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SUMMARY 


A very  important  factor  in  a high  resolution  land  combat  model  is  the 
route  selected  to  attack  enemy  units.  This  route  is  certainly  related  to  the 
positions  of  detected  surviving  enemy  weapons  since  the  attack  route  is  usually 
chosen  to  seek  cover  from  enemy  weapons  until  the  enemy  is  to  be  engaged  with 
destructive  fires.  Thus,  the  attack  route  is  affected  by  random  variables  in 
a high  resolution  simulation  and  can  not  be  prespecified  as  input  data.  Accord- 
ingly, DYNTACS  X has  a route  selection  for  helicopters  in  attacking  an  enemy 
target  complex.  However,  this  model  is  essentially  an  adaptation  of  the  ground 
unit  route  selection  model  and  has  not  been  validated.  This  report  describes 
research  conducted  to  develop  a valid  helicopter  route  selection  model  called 
DYNFLITE. 

To  begin  the  research  program,  informal  discussions  were  held  with 
pilots  to  gain  some  initial  insight  into  the  route  planning  decision  process. 
During  these  discussions,  the  pilots  stated  that  they  develop  route  plans  so  as 
to  minimize  "exposure"  to  the  enemy.  It  was  unclear  what  units  are  used  to 
measure  exposure  and  upon  what  variables  exposure  is  dependent.  However, 
it  was  suspected  that  exposure  is  somehow  related  to  a pilot's  perception  of  his 
chances  for  survival.  In  the  absence  of  other  information,  the  concept  of  "per- 
ceived threat"  was  developed  as  a measure  of  exposure  and  three  alternative 
models  to  relate  perceived  threat  to  the  variables  of  a route  selection  situation 
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were  proposed.  In  addition,  the  route  planning  task  was  formulated  as  a 
multiple  alternative  decision  problem. 


As  the  next  step,  an  experiment  was  conducted  among  qualified  helicopter 
pilots  at  Fort  Rucker  to  determine  perceived  threat  for  each  pilot  according  to 
each  of  the  three  proposed  models.  This  experiment  is  known  as  the  Phase  I 
Experiment.  As  a parallel  effort  a route  selection  algorithm  was  developed 
that  accepts  perceived  threat  as  measured  by  any  of  the  three  models  as  input, 
and  predicts  an  attack  route  on  the  basis  of  minimum  total  route  threat.  An 
important  assumption  of  this  algorithm  is  that  the  perceived  threat  of  a complete 
route  is  the  sum  of  threats  perceived  for  individual  route  segments.  Analyses 
of  the  pilot  data  from  the  Phase  I Experiment  showed  that  the  simple  additive 
relationship  may  not  be  a satisfactory  approximation  for  one  of  the  three  per- 
ceived threat  models,  i.e. , the  Acid  Test  Model. 

Next,  another  experiment  was  conducted  among  the  Phase  I subject  group 
members  to  determine  their  actual  performance  in  a route  planning  task.  For 
convenience,  this  experiment  was  conducted  during  the  same  session  as  the 
Phase  I Experiment. 

Continuing,  the  perceived  threat  data  gathered  during  Phase  I were  pre- 
pared as  input  to  the  route  selection  algorithm.  Also,  descriptions  of  the  Phase 
I route  selection  situations  were  prepared  as  input  to  the  algorithm.  For  each 
subject,  the  algorithm  was  then  exercised  to  prepare  a series  of  alternative 
route  predictions  for  each  route  selection  situation  based  on  the  three  models  of 
of  threat  perception. 
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As  a final  step  in  the  procedure,  a Phase  II  Experiment  was  conducted 
in  which  members  of  the  Phase  I subject  group  at  Fort  Rucker  were  asked  to 
rate  the  predicted  alternatives  relative  to  the  routes  they  planned  during  Phase  T. 
Data  gathered  during  this  experiment  were  then  used  not  only  to  evaluate  the  route 
selection  model  and  the  models  of  perceived  threat,  but  also  to  study  the  charac- 
teristics of  the  underlying  decision  process  itself. 

The  results  indicate  that  considerable  variation  exists  between  subjects 
so  that  it  may  be  impossible  to  develop  a single  route  selection  model  that  is 
completely  acceptable  to  all  individuals.  However,  the  decision  behavior  of  a 
predominant  class  of  decision-makers  is  apparently  described  to  an  acceptable 
degree  by  the  route  planning  model,  if  the  threat  perception  model  known  as 
Model  One  is  used  to  measure  perceived  threat.  Unfortunately,  the  model  does 
not  describe  decision  behavior  equally  well  in  all  route  selection  situations. 

Model  One  is  completely  insensitive  to  differences  among  the  pilots,  and  it 
assumes  that  the  route  threat  is  the  sum  of  the  exposure  times  to  each  enemy 
weapon.  More  experimentation  was  required  to  resolve  this  question  as  well  as 
othe  rs . 

Another  important  experiment  was  conducted  at  Fort  Ord  using  experi- 
enced pilots  who  had  also  been  trained  in  helicopter  tactics  in  a mid-intensity 
combat  environment.  This  experiment  at  Fort  Ord  was  conducted  in  the  same 
format  as  the  experiment  at  Fort  Rucker;  however,  the  results  showed  a strong 
preference  for  a different  model,  i.e. , the  Acid  Test  Model,  than  Model  One. 

The  Acid  Test  Model  is  personalistic  in  the  sense  that  pilot  inputs  are  required 
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to  specify  perceived  threat  as  a function  of  enemy  weapon  distance,  azimuth, 
and  exposure  time. 


The  conclusion  drawn  from  both  of  these  experiments  is  that  DYNFLITE 
should  use  the  Acid  Test  Model  to  measure  perceived  threat.  Also,  considerable 
variation  exists  in  the  pilot  preferences  for  the  model  routes  as  compared  to  a 
manually  selected  route.  The  experiment  revealed  cases  where  the  pilot  pre- 
ferred the  model  route  and  cases  where  a manually  constructed  route  was  pre- 
ferred. On  the  average,  the  manual  routes  were  preferred  but  the  preference 
was  mild. 
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RESEARCH  OBJECTIVES  AND  PLAN 
by  Gordon  M.  Clark 

| 1 

Introduction 

The  main  purpose  of  the  research  described  in  this  report  is  to  develop 
a model,  called  DYNFLITE,  to  predict  the  route  selected  by  a helicopter  pilot 
in  attacking  a target.  This  model  should  be  capable  of  use  in  DYNTACS  X 
(Clark  and  Hutcherson,  1971),  a simulation  of  engagements  among  battalion- 
sized combat  units  involving  ground  and  helicopter  units.  Also,  the  development 
of  DYNFLITE  provides  spillover  benefits  by  giving  insights  into  the  tactical  de- 
cision process  and  the  training  of  combat  pilots.  The  motivation  for  this  re- 
search is  that  DYNTACS  X already  has  a route  selection  model  for  helicopters; 
however,  the  model  for  comparing  alternative  routes  has  not  been  validated  and 
methods  for  soliciting  input  parameters  from  qualified  pilots  require  develop- 
ment. As  currently  formulated,  the  model  is  based  upon  hypotheses  derived 
from  current  doctrine  and  discussions  with  aerial  vehicle  pilots  with  operational 
experience.  However,  there  is  no  experimental  or  operational  evidence  to 
support  the  models  at  this  time.  Carefully  planned  experiments  need  to  be  con- 
ducted to  identify  the  underlying  decision  processes  and  to  verify  the  decision 
concepts  involved. 
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Research  Objectives 


Thus,  the  desired  output  from  this  research  is  a method 
for  obtaining  tactical  decision  parameter  inputs  and  a corresponding  model  for 
use  in  DYNTACS  X.  These  inputs  and  model  should  be  capable  of  predicting  the 
attack  route  in  diverse  situations  likely  to  be  encountered  in  the  combat  simula- 
tion. The  input  tactical  parameters  should  specify  the  tactical  doctrine  to  be 
employed  in  selecting  the  attack  route. 

In  order  to  develop  this  model,  decision  and  value-theory  concepts  will 
be  used  as  elements  of  the  route  selection  model.  Thus,  an  intermediate  objec- 
tive is  to  determine  whether  decision  and  value -theory  concepts  can  be  used  to 
explain  the  behavior  of  a military  decision-maker,  viz. , a helicopter  pilot,  in 
the  face  of  a decision  under  pressure. 

DYNTACS  X Helicopter  Route  Selection  Model 

A brief  overview  of  the  DYNTACS  X attack  route  selection  model  developed 
by  Clark  and  Hutcherson  (1971)  is  provided  as  background  to  the  research  de- 
scribed in  this  report.  This  discussion  assumes  that  a helicopter  unit,  simu- 
lated by  DYNTACS  X , has  been  requested  to  attack  a target  complex  by  a ground 
unit. 

Three  different  phases  of  the  helicopter  route  selection  problem  are  rep- 
resented by  DYNTACS  X.  These  phases  are: 

1.  selection  of  a cross  country  route  to  the  target  complex  area, 

2.  selection  of  a target  search  route,  and 

3.  selection  of  a target  attack  route. 


The  model  for  phase  1 involves  the  use  of  a modified  version  of  the  ground  route 
selection  model  (Clark,  1969b).  Thus,  the  assumptions  are  that  the  ground  route 
selection  procedure  is  valid  and  that  helicopter  pilots  react  in  the  same  fashion 


as  ground  unit  commanders  with  respect  to  cross  country  route  selection  prob- 
lems. Of  course,  the  helicopter  route  selection  model  can  accept  decision 
variables  that  represent  the  unique  tactics  of  helicopters,  but  the  model  struc- 
ture is  basically  the  same. 

The  models  for  phases  2 and  3 above  use  procedures  that  are  unique  to 
the  aerial  vehicle  route  selection  process.  Target  search  routes  are  selected 
as  follows: 

1.  define  the  mission  operations  area  as  a circle  of  specified  radius 
centered  at  the  reported  target  location, 

2.  choose  a point  whose  coordinates  are  distributed  over  the  circle 
according  to  a conical  distribution,  and 

3.  continue  selecting  points  until  an  adequate  number  of  points  to  de- 
scribe the  route  have  been  selected. 

The  rationale  for  the  search  route  selection  procedure  is  as  follows. 
First,  it  is  assumed  that  pilots  searching  for  targets  tend  to  sequentially  inves- 
tigate potential  target  locations.  Second,  no  theoretical  basis  exists  for  deter- 
mininistically  determining  how  a pilot  might  identify  potential  target  locations; 
however,  one  might  postulate  that  these  positions  would  be  more  concentrated 
in  the  vicinity  of  the  reported  target  position.  Therefore,  a conical  distribution 
is  selected  as  being  representative  of  the  process. 

I 
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Several  different  solutions  to  the  target  attack  route  selection  problem 


exist.  In  any  case,  the  attack  phase  starts  after  the  target  has  been  detected 
(by  the  attacking  helicopter  or  a scout)  and  a decision  to  attack  has  been  made. 

The  original  model  (Clark  and  Hutcherson,  1971)  employed  a running  fire  attack 
where  a route  is  drawn  through  points  starting  at  the  current  helicopter  position 
and  leading  toward  the  target  position  until  the  weapon  can  be  fired.  After  an 
initial  attack,  subsequent  attacks  using  running  fire  can  be  made  using  routes 
leading  toward  the  target  position  from  directions  that  are  randomly  selected 
but  subject  to  the  constraint  that  they  originate  from  the  "friendly  side."  Modi- 
fications by  Army  agencies  (MICOM  and  CACDA)  have  added  hover  fire  attacks 
using  "pop  up"  tactics;  moreover,  these  hover  fire  attacks  can  be  coordinated 
with  target  acquisition  activities  of  scout  helicopters. 

Scenario  for  this  Research 

A specific  tactical  scenario  has  been  selected  for  this  research  that  pro- 
vides a setting  consistent  with  the  possible  situations  represented  by  DYNTACS  X. 
The  intent  of  this  scenario  is  to  capture  the  essential  elements  of  the  multitude 
of  situations  represented  by  DYNTACS  X and  provide  an  experimental  setting 
that  is  consistent  with  the  level  of  effort  provided  for  this  research.  The  primary 
requirement  is  that  a valid  route  selection  model  for  this  scenario  would  provide 
confidence  in  the  validity  of  DYNTACS  X while  using  this  same  route  selection 
model . 

The  problem  commences  with  the  helicopter  on  the  ground  some  distance 
from  the  point  of  conflict.  A request  for  aerial  fire  support  is  received  by 
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radio,  and  this  request  contains  all  the  information  needed  by  the  pilot  in  plan- 
ning his  attack.  Prior  to  take-off  the  pilot  sketches,  on  a map  at  his  knee,  the 
location  of  the  target  and  any  other  known  enemy  weapons  in  the  vicinity.  He 
also  sketches  the  route  to  be  followed  and  the  probable  point  at  which  he  will 
deliver  his  ordnance  against  the  target.  The  route  plan  is,  of  course,  tentative 
and  may  change  as  the  mission  actually  develops.  Nevertheless,  it  is  the  initial 
plan  that  is  of  interest  in  the  present  research.  In  performing  the  planning  task 
the  pilot  is  under  pressure  to  render  a speedy  decision.  He  also  prepares  the 
plan  realizing  that  he  will  face  a hostile  opponent  while  executing  the  plan.  Con- 
sequently, this  decision  task  has  all  the  elements  of  an  attack  upon  an  enemy 
target  position  with  the  exception  of  the  search  for  the  target. 

Some  Benefits 

Research  in  the  route  planning  decision  process  is  of  importance  for 
several  reasons.  First,  route  planning  is  a fundamental  and  very  common  de- 
cision process  of  combat,  both  for  ground  vehicles  and  for  aerial  vehicles.  A 
valid  model  to  explain  decision  behavior  in  this  task  would  be  a contribution  to  a 
better  understanding  of  combat  processes  in  general.  Also,  military  planners 
have  made  extensive  use  of  simulations  to  describe  combat  operations  in  order 
to  study  the  effects  of  variations  in  weapon  system  characteristics,  tactics  and 
doctrine^  on  the  operational  performance  of  combat  units.  Any  detailed 

■^The  following  definitions  are  offered  from  Webster's  New  International 
Dictionary,  Second  Edition:  Tactics  1.  The  science  or  art  of  disposing  and  ma- 

I neuvering  troops  or  ships  in  action  or  in  the  presence  of  the  enemy.  2 

Doctrine  1 2.  A principle  or  position,  or  the  body  of  principles  in  any 

branch  of  knowledge. . . . 
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simulation  of  this  type  requires  that  one  be  able  to  predict  the  routes  that  are 


selected  by  combatants.  For  example,  Clark  (1969b)  shows  that  the  routes 
selected  by  armored  vehicles  can  affect  the  intervisibility  relationships  that 
exist  between  them  and  thus  affect  their  ability  to  fire  on  one  another.  Also, 
characteristics  of  the  terrain  surface  in  the  vicinity  of  a vehicle  can  change 
drastically  as  routes  are  changed,  thereby  producing  effects  on  the  mobility  of 
the  vehicle.  However,  it  is  not  enough  to  simply  prespecify  the  routes  that 
combatants  will  follow,  for  the  decision-maker  selects  these  routes  in  combat 
to  take  advantage  of  the  known  disposition  of  the  enemy,  and  this  knowledge 
changes  probabilistically  as  a function  of  time. 

Next,  the  research  may  very  well  produce  significant  practical  benefits 
in  terms  of  insights  into  the  decision  process  itself.  For  example , it  may  be 
that  a fundamental  limitation  on  the  performance  of  a pilot  in  the  route  planning 
task  is  his  inability  to  accurately  perceive  all  the  important  variables  of  the 
situation.  One  result  would  be  that  he  feels  he  is  performing  according  to  pre- 
scribed doctrine  when  in  fact  he  is  not.  If  this  were  the  case,  the  performance 
of  the  decision-maker  might  be  more  important  as  a limit  on  unit  operational 
performance  than  either  prescribed  doctrine  or  the  physical  capabilities  of  the 
unit's  weapons.  Such  a result  would  indicate  a need  for  some  sort  of  aid  in  the 
route  planning  task.  Also,  such  behavior  in  the  route  planning  task  most  prob- 
ably would  indicate  the  existence  of  the  same  phenomenon  in  other  types  of 
military  decisions  under  pressure. 
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Another  insight  would  be  provided  if  it  were  discovered  that  two  or  more 
distinct  and  consistent  patterns  of  decision  behavior  exist  among  pilots.  Such 
a finding  would  indicate  that  it  may  be  impossible  to  develop  one  universally 
acceptable  model  of  the  decision  process.  The  design  philosophy  of  detailed 
combat  operations  simulations  would  be  drastically  altered  by  this  finding. 

Also,  if  one  or  more  of  the  patterns  of  behavior  were  considered  to  yield  poor 
combat  results  then  a change  in  training  syllabi  or  in  pilot  selection  procedures 
might  be  indicated. 

Another  insight  would  be  provided  if  it  were  found  that  certain  pilots  are 
more  consistent  in  their  decision  performance  than  others.  For  example,  when 
faced  repeatedly  with  the  same  tactical  situation,  some  pilots  might  almost  al- 
ways select  essentially  the  same  attack  route  while  others  might  select  routes 
that  vary  extensively  from  one  trial  to  the  next.  Would  such  a finding  indicate 
simple  psychological  differences  between  pilots  or  could  such  differences  be 
traced  to  variations  in  experience?  Was  the  formal  training  experienced  by  the 
pilots  the  same?  Were  the  pilots  exposed  to  essentially  the  same  enemy  for 
about  the  same  amount  of  time?  Numerous  research  programs  could  be  pursued 
to  study  variations  in  decision  performance  of  the  type  described  above. 

Next,  in  much  of  the  above  discussion  it  was  implied  that  the  route  plan- 
ning research  program  could  yield  insights  into  the  decision  process  that  might 
be  constructively  incorporated  into  the  procedures  used  to  select  and  train  pilots. 
However,  a valid  model  of  the  route  planning  process  could  be  used  directly  in 
the  pilot  training  program  itself.  For  example,  the  model  could  be  used  in 
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studies  to  isolate  good  decision  principles , perhaps  on  the  basis  of  an  objectively 
determined  survival  probability.  The  model  could  also  be  used  as  a device  to 
train  decision  makers  in  the  application  of  good  decision  principles  and  to  mea- 
sure their  progress  in  the  training  program.  Thus,  the  route  planning  model 
has  a broad  potential  for  use  in  training. 

Some  Related  Research 

A review  of  the  literature  concerning  military  decision-making  under 
pressure  to  assess  available  methodology  for  modeling  route  selection  only  con- 
firms the  need  for  more  research  in  this  area.  This  review  included  methodol- 
ogy for  determining  when  to  fire  at  enemy  weapons  and  what  target  to  select  in 
order  to  take  advantage  of  available  methodology.  Studies  of  this  important  de- 
cision process  are  very  limited  in  number.  Nevertheless,  two  examples  were 
found  whose  results  merit  examination. 

Olson  (1970)  performed  research  to  determine  the  firing  decision  behav- 
ior of  a tank  commander  in  a stationary  firing  position  when  confronted  with  an 
enemy  tank  that  is  also  stationary.  In  the  situation  studied  it  was  possible  that 
the  decision  to  fire  could  result  in  the  enemy  tank  being  destroyed  before  being 
able  to  return  fire.  However,  it  was  also  possible  that  firing  resulting  in  a miss 
could  motivate  the  enemy  to  fire. 

Independent  variables  in  the  study  were  target  range,  the  degree  of  cover 
provided  the  target  tank,  the  aspect  of  the  target  and  previous  engagement  history. 
It  was  found  that  the  decision  to  fire  or  not  fire  could  be  predicted  quite  well  from 
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a knowledge  of  the  independent  variables  included  in  the  study.  Tank  command- 
ers were  found  to  be  quite  conservative  in  this  decision  context  in  that  they 
tended  to  fire  as  soon  as  they  felt  that  firing  would  decrease  their  chances  of 
being  hit  by  the  enemy.  With  his  model,  Olson  attempted  to  prodict  the  range 
at  which  all  tank  commanders  would  fire.  He  then  assumed  that  the  proportion 
of  the  population  that  would  fire  at  any  greater  range  could  be  predicted  as  a 
linear  function  of  range.  Unfortunately,  predictions  based  on  this  assumption 
were  not  as  accurate  as  desired. 

Parry  (1971)  extended  Olson's  work  to  treat  the  case  of  a fixed  firer, 
moving  target.  The  effects  of  eleven  independent  situation  variables  on  four 
response  variables  were  treated  in  this  study  by  using  a 1/16  replicate  of  a 211 
factorial  design.  All  of  the  response  variables  were  related  to  detection  and 
engagement.  A very  interesting  aspect  of  the  study  was  the  analysis  required 
for  such  a complex  experimental  design.  However,  one  very  important  insight 
into  the  decision  process  was  also  determined.  It  was  found  that  the  tank 
commander's  decision  behavior  is  a very  complex  function  of  the  situation 
variables.  Indeed,  most  of  the  variation  in  the  response  variables  was  found 
to  be  attributable  not  to  basic  variations  in  the  independent  variables  themselves 
but  to  interactions  among  the  variables.  For  example,  the  turret  position  of 
the  enemy  tank  coupled  with  the  movement  history  of  the  tank  might  lead  the 
commander  to  believe  that  the  enemy  has  detected  his  position. 

The  two  studies  discussed  above  involved  one  or  more  decisions  in  which 
two  alternatives  exixted.  The  number  of  studies  of  military  decision  processes 
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involving  the  selection  of  an  alternative  from  a multiplicity  of  alternatives  is 
more  limited  yet.  First,  let  it  be  noted  that  had  Olson  and  Parry  extended  their 
work  to  treat  multiple  targets,  then  a multiple -alternative  decision  process 
would  have  resulted.  The  decision  would  not  only  have  been  whether  to  fire  but 
also  at  which  target.  Both  Olson  and  Parry  demurred  from  such  an  extension 
based  on  the  complexity  of  the  processes  involved. 

Next  the  review  considered  available  route  selection  models,  and  several 
of  these  models  will  not  be  outlined. 

One  of  the  earliest  representations  of  the  military  route  selection  pro- 
cess appeared  in  a combat  simulation  known  as  CARMONETTE  . As  described 
by  Zimmerman  and  Kraft  (1957),  the  route  selection  model  uses  the  concept  of 
tactical  areas  to  define  potential  assault  routes  for  a tank  company  and  its  sup- 
porting weapons.  For  analysis  purposes,  the  number  of  possible  assault  routes 
is  limited  to  approximately  ten.  The  tactical  areas  are  specified  by  classifying 
the  terrain  into  irregularly  shaped  areas  such  as  hilltops,  valleys,  forests, 
fields  and  villages.  Thus,  each  area  is  approximately  homogeneous  with  respect 
to  an  important  environmental  characteristic. 

Once  the  potential  assault  routes  are  identified,  each  is  evaluated  to  de- 
termine the  casualties  and  travel  time  that  the  company  would  expect  to  experi- 
ence if  the  route  were  actually  followed.  In  turn,  these  two  measures  are  com- 
bined through  a weighting  procedure  to  produce  a relative  "desirability"  for  each 
route.  Finally,  the  route  possessing  the  greatest  "desirability"  is  selected  for 
implementation. 

10 
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The  model  described  above  employs  several  important  concepts  which 
have  been  exploited  in  the  helicopter  route  planning  model  described  in  Chapter 


2.  For  example,  the  environmental  characteristics  of  the  terrain  are  considered 
in  developing  a set  of  potential  routes.  Also,  a measure  is  developed  with  which 
to  evaluate  each  potential  route,  and  this  measure  is  sensitive  to  the  variables 
describing  the  tactical  situation.  Unfortunately,  the  route  selection  procedure 
has  never  been  validated  as  a descriptive  model. 

A later  version  of  CARMONETTE  contains  an  entirely  different  route 
selection  model  and  is  described  by  Adams,  et  al.  (1961).  The  terrain  is 
organized  into  a 36  x 36  array  of  equal  area  grid  squares  with  the  environmental 
characteristics  of  each  square  quantified  by  a set  of  descriptive  codes.  An  ele- 
ment can  occupy  any  position  on  the  battlefield,  and  its  local  environment  can  be 
determined;  however,  the  position  of  the  element  is  determined  only  to  the  center 
of  the  nearest  grid  square.  Other  significant  features  of  the  model  may  be  sum- 
marized as  follows: 

1.  The  movement  trace  of  an  element  (e.g. , a tank)  is  represented 
as  a series  of  connected  line  segments  extending  between  adjacent 
grid  square  centers. 

2.  An  element  plans  its  route  by  examining  the  desirability  of 
traveling  to  each  of  the  eight  grid  squares  adjacent  to  the  present 
grid  square. 

3.  The  movement  decision  is  performed  by  a Monte  Carol  Move  Se- 
lection Model. 

Again,  this  model  has  not  been  validated  as  a descriptive  route  planning  model. 

Another  route  selection  procedure  which  was  obviously  influenced  in  de- 
sign by  the  second  CARMONETTE  model  is  that  which  is  incorporated  into  the 


11 


Signal  Corps  ground  Combat  Simulator.  This  model  was  summarized  by 
USACDC  (1963)  and  is  a division-level  simulator  emphasizing  information  and 
communication  flow.  Instead  of  treating  individual  vehicles,  the  model  de- 
scribes the  activities  of  a tactical  unit. 

The  terrain  is  organized  into  grid  squares , and  each  square  is  given  a 
set  of  indices  specifying  the  local  environment.  In  turn,  these  indices  are  used 
in  the  route  selection  procedure  to  compute  a grid  terrain  index  which  repre- 
sents the  "cost"  of  leaving  a grid.  No  research  has  been  performed  to  deter- 
mine how  well  the  model  performs  as  a descriptor  of  the  tactical  unit  route 
selection  process. 

The  route  selection  model  having  the  most  influence  upon  this  research 
is  a model  used  to  select  routes  for  ground  units  in  DYNTACS  X (Clark,  1966). 
Recall  that  the  existing  DYNTACS  X helicopter  route  selection  model  is  a modi- 
fication of  this  model.  The  ground  route  selection  model  uses  a dynamic  pro- 
gramming algorithm  to  select  a route  of  "least  tactical  difficulty"  through  a 
route  selection  grid  laid  down  along  a specified  axis  of  advance . The  area 
covered  by  the  grid  is  known  as  the  "route  selection  area"  and  represents  the 
decision  maker's  spatial  planning  horizon.  Many  elements  of  this  model  have 
been  used  as  a basis  for  the  model  to  be  discussed  in  Chapter  2. 

In  addition,  research  has  been  conducted  by  Captain  Donald  Shive  (1976) 
for  his  Master's  Thesis  at  The  Ohio  State  University  to  validate  the  DYNTACS  X 
ground  route  selection  model.  Shive  solicited  inputs  for  the  model  from  experi- 
enced army  officers  at  Fort  Knox,  tested  model  assumptions  using  these  inputs, 
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and  attempted  to  predict  routes  selected  by  these  officers  by  the  route  selection 
model.  The  basic  model  assumptions  were  verified;  however,  several  exten- 
sions or  modifications  to  the  model  were  recommended  by  Shive  to  improve  its 
predictability. 

A final  example  of  a descriptive  model  of  the  military  route  selection 
process  is  incorporated  into  the  SIAF  model  developed  by  TRW  (1971).  This 
procedure  was  established  to  represent  the  route  selection  activities  of  a small 
dismounted  patrol.  Again,  the  terrain  is  divided  into  grid  squares  and  a dynamic 
programming  algorithm  is  employed  to  select  the  optimal  route.  The  route  con- 
sists of  a series  of  connected  line  segments  joining  adjacent  grid  points,  but 
each  segment  is  evaluated  on  the  basis  of  its  "utility."  This  "utility"  value  is 
determined  as  a linear  combination  of  terms  relating  to  such  factors  as  the  prob- 
ability of  detecting  the  enemy,  the  probability  of  being  detected  and  the  difficulty 
of  movement  along  the  segment.  Certain  grid  points  can  be  identified  as  posi- 
tions to  be  avoided  (for  protection)  or  as  positions  to  be  approached  (for  observa- 
tion). Other  grid  points  can  be  identified  as  positions  through  which  a route  can- 
not pass  under  any  circumstance.  These  points  are  most  representative  of  ob- 
stacles to  movement.  No  attempt  to  validate  the  SIAF  route  selection  model  has 
been  made. 

In  conclusion,  little  research  has  been  conducted  to  understand  even  the 
decision  when  to  open  fire  which  is  a simple  binary  go-no  go  military  decision 
under  pressure.  For  the  more  complex  route  selection  decision  which  is  a 
pressured  military  decision  involving  multiple  alternatives,  much  less  is  known. 
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Although  several  models  have  been  proposed,  little  research  has  been  per- 
formed to  validate  them. 


The  Research  Plan 

At  the  outset  of  the  research  program,  informal  discussions  were  held 
with  pilots  to  gain  some  initial  insight  into  the  route  planning  decision  process. 
During  these  discussions , the  pilots  stated  that  they  develop  route  plans  so  as 
to  minimize  "exposure"  to  the  enemy.  It  was  unclear  what  units  are  used  to 
measure  exposure  and  upon  what  variables  exposure  is  dependent.  However, 
it  was  suspected  that  exposure  is  somehow  related  to  a pilot's  perception  of  his 
chances  for  survival.  In  the  absence  of  other  information,  the  concept  of  "per- 
cevied  threat"  was  developed  as  a measure  of  exposure  and  three  alternative 
models  to  relate  perceived  threat  to  the  variables  of  a route  selection  situation 
were  proposed.  In  addition,  the  route  planning  task  was  formulated  as  a multiple 
alternative  decision  problem. 

As  the  next  step,  experiments  were  conducted  among  qualified  helicopter 
pilots  to  determine  perceived  threat  for  each  pilot  according  to  each  of  the  three 
proposed  models.  This  experiment  is  known  as  the  Phase  I Experiment  (Chapter 
3).  As  a parallel  effort  a route  selection  algorithm  was  developed  that  accepts 
perceived  threat  as  measured  by  any  of  the  three  models  as  input  and  predicts 
an  attack  route  on  the  basis  of  minimum  total  route  threat  (Chapter  2). 

Next,  other  experiments  were  conducted  among  the  Phase  I subject 
group  members  to  determine  their  actual  performance  in  a route  planning  task. 
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For  convenience,  these  experiments  were  conducted  during  the  same  sessions 


I 


as  the  Phase  I Experiments . 

Continuing,  the  perceived  threat  data  gathered  during  Phase  I were  pre- 
pared as  input  to  the  route  selection  algorithm.  Also,  descriptions  of  the  Phase 
I route  selection  situations  were  prepared  as  input  to  the  algorithm.  For  each 
subject,  the  algorithm  was  then  exercised  to  prepare  a series  of  alternative 
route  predictions  for  each  route  selection  situation  based  on  the  three  models  of 
threat  perception. 

As  a final  step  in  the  procedure,  Phase  II  Experiments  (Chapters  4,5, 
and  6)  were  conducted  in  which  members  of  the  Phase  I subject  group  were  asked 
to  rate  the  predicted  alternatives  relative  to  the  routes  they  planned  during 
Phase  I.  Data  gathered  during  these  experiments  have  been  used  not  only  to 
evaluate  the  route  selection  model  and  the  models  of  perceived  threat,  but  also 
to  study  some  of  the  decision  performance  questions  outlined  in  an  earlier  section 
of  this  chapter. 

Finally,  conclusions  are  drawn  in  Chapter  7 from  these  experiments  to 
determine  a decision  model  (DYNFLITE)  for  use  in  the  DYNTACS  X route  se- 
lection model. 


I 

I 
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CHAPTER  2 

A ROUTE  PLANNING  MODEL 
by  Don  C.  Hutcherson 

I 

Introduction 

In  this  chapter  attention  is  directed  to  formulation  of  a model  of  the  route 
planning  task.  This  model  will  embody  many  decision  and  value-theory  concepts 
and  is  intended  as  a descriptive,  rather  than  a normative,  model  of  pilot  decision 
behavior  in  the  task  of  interest. 

As  a first  step,  a possible  conceptualization  of  the  decision  process 
employed  by  a pilot  in  planning  a route  is  presented.  This  conceptualization  is 
based  upon  the  assumption  that  pilots  perform  according  to  a set  of  well  known 
principles  accepted  as  elements  of  decision  and  value-theory  and  which  have 
their  counterparts  in  psychophysical  theory.1  However,  it  is  not  intended  that  a 
decision  model  be  derived  which  is  actually  a representation  of  the  thought 
process  conceptualization.  Instead,  the  decision  model  to  be  developed  is 
intended  merely  to  predict  the  final  results  of  that  decision  process.  The 
decision  process  conceptualization  is  developed  to  promote  understanding  and 
to  serve  as  a framework  of  thought  within  which  a decision  model  might  be 


developed.  The  process  conceptualization  has  not  been  validated,  whereas 


Psychophysics  in  simplest  terms,  may  be  defined  as  the  field  of  study  of 
the  experimental  psychologist  that  is  involved  with  man's  ability  to  discriminate. 
From  Galenter  (1962),  questions  that  the  psychophysicist  might  ask  are:  "What 
is  it  that  people  do  when  they  discriminate,  how  do  they  do  it,  how  can  we  under- 
stand it,  what  are  its  limits ?" 
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considerable  effort  has  been  expended  to  validate  the  decision  model  as  a 
predictor.  Chapters  3 through 6 present  a detailed  description  of  this  effort. 

The  Decision  Process 

The  route-planning  task  is  conceived  to  be  a task  conducted  under  con- 
siderable pressure  for  a speedy  decision.  Therefore , it  is  assumed  that  the 
task  is  conducted  almost  exclusively  as  an  intuitive  and  subjective  exercise 
which  yields  decisions  that  would  be  considered  less  than  optimal  even  by  the 
decisionmaker,  were  he  to  have  sufficient  time  to  conduct  a thorough  analysis 
of  the  situation.  This  assumption  is  in  agreement  with  the  hypothesis  stated  by 
Morris  (1964)  that  decision  processes  pressured  by  time  tend  toward  intuition 
and  away  from  analysis. 

Nevertheless,  the  process  does  contain  elements  that  are  common  to 
most  decision  problems . The  pilot  conducts  an  implicit  search  for  alternatives. 
He  employs  a decision  rule  to  evaluate  the  alternatives  as  they  become  apparent. 
Finally,  he  employs  a stopping  rule  to  determine  when  he  should  cease  searching 
for  new  alternatives  and  accept  one  of  the  ones  he  has  found  as  his  plan  of  action. 
This  stopping  rule  could  specify  termination  of  search  with  the  first  feasible 
alternative. 

We  are  in  agreement  with  Morris  (1964)  in  his  view  that  the  potential 
for  effective  decision-making  in  a decision  task  is  limited  during  the  first 
phase  of  the  decision  process;  i.e.,  during  the  search  for  alternatives.  It  is 
hypothesized  that  the  pilots  take  a rather  simplistic  view  of  the  decision  alter- 
natives available  to  them.  They  are  limited  by  their  capacity  to  perceive  all 
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the  complexities  of  the  situation  and  the  terrain  over  which  routes  are  to  be 
planned.  Subject  to  time  constraints,  they  successively  isolate  satisfactory 
alternative  approaches  to  attacking  the  target  and  from  this  set  they  choose 
one  they  perceive  as  best.  The  March-Simon  (1958)  hypothesis  provides  a 
basis  for  this  principle  of  route  selection: 

"Most  human  decision-making,  whether  organizational  or  individual, 
is  concerned  with  the  discovery  and  selection  of  satisfactory  alterna- 
tives; only  in  exceptional  cases  is  it  concerned  with  the  discovery 
and  selection  of  optimal  alternatives." 

Since  an  optimal  alternative  is  one  that  is  demonstrably  the  best  of  all  possible 
alternatives,  the  pilots  are  less  than  optimal  in  their  decision  behavior.  This 
view  is  certainly  confirmed  by  a mass  of  literature  concerned  with  learning  and 
other  psychophysical  phenomena. 

We  further  hypothesize  that  fairly  simple  decision  rules  are  applied  in 
isolating  and  evaluating  the  various  route  alternatives . A possible  alternative 
is  analyzed  first  to  determine  whether  it  satisfies  all  the  constraints  placed  upon 
it  by  the  facts  of  the  scenario  and  by  the  performance  characteristics  of  the 
weapon  system  being  used.  That  is,  to  be  a satisfactory  alternative  the  route 
must  permit  an  attack  upon  the  target.  Routes  passing  this  aspiration  threshold 
test  are  then  evaluated  using  simple  decision  rules  and  the  best  alternative  is 
subsequently  chosen. 

The  contention  of  the  previous  paragraph  is  certainly  well-founded  in  the 
results  of  numerous  psychophysical  studies.  Indeed,  experimenters  have  found 
that  regardless  of  the  complexity  of  a decision  bisk,  the  behavior  of  a subject 
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can  often  be  described  with  very  simple  linear  models.  Moreover,  in  complex 
decision  tasks,  subjects  tend  to  process  very  limited  amounts  of  information  in 
performing  the  tasks . This  is  not  to  say  that  subjects  recognize  their  limited 
capabilities;  rather,  this  is  the  way  people  behave.  A thorough  discussion  of 
studies  conducted  along  these  lines  and  evidence  to  support  our  contentions  are 
contained  in  the  recent  work  by  Slovic  and  Lichtenstein  (1970). 

The  Decision  Model 

From  the  discussion  of  the  previous  section  there  is  the  suggestion  that 
the  decision  model  to  be  developed  should  be  able  to  isolate  acceptable  alter- 
natives , evaluate  the  alternatives  according  to  some  measure  of  worth  (probably 
subjective)  and  yield  as  the  answer  that  alternative  among  those  examined  which 
is  judged  best  by  the  decision  rule  being  used.  The  proposed  model  does  in 
fact  possess  all  these  characteristics,  and  they  will  be  discussed  throughout 
the  remainder  of  this  chapter.  The  discussion  begins  with  a derivation  of  the 
decision  rule. 

2 

Emphasis  has  been  added  to  indicate  that  while  behavior  can  be  described 
by  such  simple  models,  the  models  in  themselves  do  not  necessarily  reveal  an 
explanation  of  the  behavior.  Fortunately,  description  is  the  primary  goal  of 
the  present  research.  However,  some  doubt  is  cast  on  the  concept  of  simplic- 
ity in  at  least  one  type  of  military  decision-making  problem  by  the  results  of 
Parry  (1971)  discussed  in  Chapter  1 (see  page  9). 
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The  Decision  Rule 


The  route  planning  task  may  be  characterized  by  a set  of  alternative 
route  plans  aj,  a set  of  payoffs  Vj  and  a set  of  probabilities  Pjj,  with  a par- 
ticular pjj  specifying  the  probability  that  Vj  is  realized,  given  that  route  plan 
aj  is  followed.  Now,  given  a route  alternative  aj,  there  are  four  possible  future 
states  Oj  that  can  exist  after  completion  of  the  mission: 

Oj  - both  the  target  and  helicopter  fail  to  survive; 

C>2  - the  target  is  dead,  but  the  helicopter  survives; 

03  - the  target  survives,  but  the  helicopter  is  killed;  and 

04  - both  the  target  and  the  helicopter  survive. 

The  payoffs  Vj  are  some  unknown  function  of  Oj ; that  is  Vj  = F(Oj) . 

The  decision  rule  is  to  provide  a principle  of  choice  between  the  alter- 
natives aj,  i = 1,  2,  . ..  . It  should  be  representative  of  that  used  by  a helicopter 
pilot  and  it  most  probably  should  utilize  the  information  provided  by  Vj  and  Pjj. 
The  preferred  alternative  selected  according  to  the  decision  rule  is  given  the 
symbol  a . 

There  are  a number  of  decision  rules  that  might  be  applied  to  select  a . 
The  decision-maker  could  view  the  decision  as  one  under  uncertainty,  one  under 
risk  or  one  under  assumed  certainty.  In  the  first  case  the  decision-maker  is 
unable  or  unwilling  to  attach  any  values  to  the  probabilities  p^ . In  the  last 
case,  it  is  assumed  that  there  is  but  one  possible  future  Oj  associated  with 
a given  route  plan  alternative  U|.  A view  that  the  decision  is  one  under  risk 
offers  a middle  ground  in  which  values  are  assignable  to  the  probabilities  p^. 
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Under  assumed  certainty,  each  route  alternative  aj  would  have  asso- 
ciated with  it  one  of  the  four  futures  CL,j  = l,  2,  3,  4.  That  is,  for  a^  if  Om 
is  assumed,  then  Pj,  m = 1,  p^j  = 0 for  j f m,  and  the  payoff  for  a^  would  be 
Vm.  Given  that  a different  value  of  m could  be  assumed  for  each  i,  i = 1,  2, . . . , 
the  decision  rule  might  be  to  select  the  plan  that  results  in  the  greatest  payoff 
V*  where  V*  = max  2 p^.  y. . Some  other  procedure  such  as  coin  tossing  could 

i j 

be  used  if  more  than  one  route  produced  the  same  future  or  if  more  than  one 
future  realized  by  the  plans  had  an  equivalent  payoff.  One  imght  suspect  that 
pilots  would  associate  with  all  routes  either  possible  future  O2  or  possible 
future  above,  and  possibly  only  future  might  be  used.  In  this  last  case, 

a lot  of  coin  tossing  would  be  involved  in  arriving  at  a decision  since  a unique 

* * 
value  for  a would  never  be  produced  by  the  decision  rule  employing  V . 

Under  uncertainty,  several  decision  rules,  or  principles  of  choice,  have 
been  proposed.  After  Morris  (1964),  notable  examples  are:  the  Laplace  Prin- 
ciple, the  Maximin  Principle,  the  Hurwicz  Principle  and  the  Savage  Principle. 

In  our  context  the  variation  among  these  principles  again  lies  in  the  assumptions 
made  about  Pjj . 

The  Laplace  Principle  simply  states  that  one  should  select  the  alterna- 
tive which  maximizes  expectation  assuming  equally  likely  futures.  In  the  prob- 
lem at  hand,  the  principle  yields  nonsensical  results  in  that  all  p..  assume  the 
same  value.  The  principle  docs  not  lead  to  a choice  since  all  alternatives  have 
the  same  value.  The  only  way  the  principle  could  lead  to  a choice  would  be  for 
some  of  the  p^  values  to  be  a priority  set  to  zero.  But  then  this  would  be  a 
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departure  from  the  Laplace  Principle.  The  principle  also  has  a fundamental 
weakness  in  the  problem  at  hand  for  the  pilots  probably  do  not  think  of  02  and 
O4  as  having  probabilities  equivalent  to  Oj  and  O3. 

The  Maximin  Principle  states  that  one  should  select  the  alternative  that 
maximizes  the  minimum  payoff.  However,  in  the  problem  at  hand,  the  mini- 
mum payoff  is  the  same  for  all  alternatives  so  the  principle  does  not  lead  to  a 
decision.  Also,  the  principle  is  probably  much  too  conservative  considering 
the  differences  between  02,  O4  and  Oj,  03. 

Unmodified,  the  Hurwicz  Principle  fails  to  lead  to  a decision  in  our 

decision  problem  for  the  same  reason  that  the  Maximin  Principle  failed. 

However,  it  can  be  modified  and  used.  The  modified  principle  is  defined  as 

follows : For  each  alternative  aj  define  an  index  of  optimism  ctj  that  expresses 

the  degree  of  optimism  the  decision-maker  feels  for  the  alternative.  If  he  is 

extremely  optimistic  setoq  = 1.  If  he  is  extremely  pessimistic  set  04  = 0.  Then 
* 

select  as  a the  alternative  aj  having  the  maximum  value  of  c^.  Of  course,  this 
principle  of  choice  is  no  more  than  the  principle  that  has  always  been  used  for 
intuitive  decision-making.  Consequently,  the  principle  really  contributes  nothing 
new  in  the  present  context. 

The  Savage  Principle  operates  on  a quantity  known  as  regret  and  defined 
as  the  difference  between  a given  payoff  and  the  maximum  payoff  that  could  be 
realized  for  the  possible  future  under  consideration.  The  principle  fails  as  a 
principle  of  choice  for  the  same  reason  that  the  Maximin  Principle  and  the 
unmodified  llurwicz  Principle  failed. 
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Considering  the  deficiencies  outlined  above,  it  appears  that  the  route 
planning  task  as  it  has  been  formulated  cannot  be  structured  as  a classical 


decision  under  assumed  certainty  or  as  one  under  uncertainty.  We,  therefore, 
turn  to  a view  of  the  decision  as  one  under  risk. 

Edwards  (1955)  has  described  four  models  which  are  related  and  which 
have  been  used  extensively  in  studies  of  risky  decision-making  behavior.  These 
models  assume  that  a decision  maker  will  choose  the  alternative  that  max- 
imizes his  expected  payoff,  whether  the  payoff  be  measured  objectively  or  sub- 
jectively. Expectation  in  these  models  is  computed  using  both  objective  and  sub- 
jective probabilities. 

The  first  model  is  called  the  expected  value  (EV)  model,  with  the 
expected  payoff  computed  using  objective  measures  for  both  payoff  and  probabil- 
ity. Such  a model  would  be  of  use  in  the  present  context  only  if  the  payoffs  Vj 
and  the  probabilities  p- j were  measured  in  an  objective  fashion.  Since  the 
decision  rule  is  to  represent  a basically  subjective  process  we  reject  the  EV 
model  in  lieu  of  more  subjectively  based  candidates. 

The  next  model  computes  expected  payoff  with  subjective  probabilities 
and  objective  payoff  measures.  The  expected  payoff  is  called  subjectively 
expected  money  (SEM).  Such  a model  is  more  appropriate  to  our  needs  than  the 
EV  model  in  that  it  becomes  more  sensitive  to  variations  across  individuals 
through  the  use  of  subjective  probabilities.  If  an  objective  payoff  measure  such 
as  death  of  the  target  or  survival  of  the  helicopter  were  used,  then  the  model 
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would  be  exactly  suited.  For  example,  we  could  use  ) Vj  - F(Oj)l 

I N J 

as  the  components  of  the  model  where  the  Pjj  are  measured  subjectively  and 

F(Oj)  = fl  if  j = 2,4 
|0ifj  = l,3. 

In  this  model,  the  only  matter  of  concern  is  survival  of  the  helicopter. 

The  expected  utility  (EU)  model  employs  objective  probabilities  in  com- 
puting expected  payoff,  but  the  payoff  is  measured  by  utility  (subjective  value) . 
This  model  would  be  applicable  in  the  present  context  if  objective  probabilities 
were  appropriate,  and  if  the  utility  function  V-  = F(Oj ) could  be  determined. 

It  is  another  personalistic  model,  but  this  time  the  sensitivity  to  individual 
differences  arises  through  the  subjectivity  of  the  payoff.  The  EU  model  is 
not  as  applicable  in  the  present  context  as  the  SEM  model  because  of  its  use 
of  objective  probabilities.  While  one  might  visualize  the  existence  of  an 
objective  measure  of  payoff  as  was  done  for  the  SEM  model  it  is  difficult  to 
perceive  how  objective  probabilities  could  be  determined  by  a pilot  planning 
an  attack. 

Finally,  the  subjective  expected  utility  (SEU)  model  employs  both 
subjective  probabilities  and  subjective  values  to  measure  expected  payoff. 

This  model  is  the  most  sensitive  to  variations  among  decision  makers,  and 
from  this  point  of  view  might  be  the  most  desirable  model  of  the  four  for  our 
use . It  is  also  the  most  general  model  for  two  reasons . First , it  could  turn 
out  that  subjective  probabilities  closely  approximate  objective  probabilities. 
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Second,  the  utility  function  F(CF)  could  very  well  approximate  an  objective 

function  such  as  that  used  in  the  example  of  the  SEM  model.  Thus,  all  the 

previously  cited  models  could  conceivably  be  accommodated  within  the  SEU 

model  representation.  To  use  the  SEU  model  one  would  have  to  determine  die 

utility  function  F(Oj)  and  the  subjective  probabilities  p^  for  each  alternative  af. 

One  would  then  select  a * as  the  alternative  associated  with 

4 
Z 

j=l 

A potential  difficulty  underlying  the  use  of  all  the  cited  risky  decision 
models  in  predicting  human  behavior  is  that  the  expectation  maximization  decision 
rule  is  sometimes  a fairly  poor  predictor  of  apparently  idiosyncratic  behavior. 
There  is  the  phenomenon,  called  risk  by  many  (unfortunately),  that  apparently 
enters  the  decision  process  and  which  numerous  authors  have  blamed  for  the 
poor  showing  of  the  decision  rule.  For  example,  in  a gambling  decision  in 
which  the  utility  for  money  has  been  determined  for  an  individual,  and  in  which 
subjective  probabilities  have  been  elicited,  expectation  maximization  docs  not 
predict  the  gambler's  decision.  He  may  elect  not  to  gamble  if  the  stakes  are 
too  high  or  if  the  probability  of  winning  is  too  low,  even  though  the  expected 
utility  of  gambling  is  greater  than  that  for  sitting  tight. 

A variety  of  theories  and  near  theories  have  been  proposed  to  correct 
for  risk.  Perhaps  the  oldest  treatment  is  incorporated  in  the  "expectation- 
variance"  principle  embodied  in  most  introductory  discussions  of  utility  models. 
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This  principle  states  that  a decision  maker  faced  with  several  alternatives 
will  act  so  as  to  maximize  his  expected  utility.  That  is,  as  stated  previously, 
he  will  choose  that  alternative  that  yields  maximum  expected  payoff  with  pay- 
off measured  in  terms  of  utility.  However,  faced  with  two  or  more  alternatives 
to  which  he  is  otherwise  indifferent,  he  will  choose  that  alternative  that  exhibits 
the  least  variance  in  outcome.  For  example,  consider  the  two  alternative 
gambles  A and  B described  by  the  following  decision  matrix. 


The  two  alternatives  are  equivalent  with  respect  to  expected  payoff  ($0 ),  but 
the  expectation- variance  principle  predicts  that  the  decision  maker,  neverthe- 
less, would  choose  alternative  B over  alternative  A since  the  variance  of  alter- 
native B is  $1.  00  while  the  variance  of  alternative  A is  $4.  00.  Thus,  the  ex- 
pectation-variance principle  takes  the  variance  in  outcome  of  an  alternative  as 
a measure  of  the  risk  of  that  alternative.  However,  as  stated  by  Pollatsek  and 
Tversky  (1969),  empirical  evidence  for  systematic  variance  preferences  is 
somewhat  inconclusive. 
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Other  approaches  to  the  study  of  risk  in  decision  making  have  been  made 
by  Coombs  and  his  associates  (cited  extensively  by  Pollatsek  and  Tversky,  1969 
and  by  Wendt,  1970).  These  studies  have  explored  the  variables  affecting  the 
perception  of  risk  and  have  also  dealt  with  the  question  of  how  perceived  risk 
affects  preferences.  The  theory  postulates  that  each  individual  has  a most 
preferred  risk  level  and  that  instead  of  trying  to  minimize  variance  in  outcome 
one  tries  to  minimize  the  deviation  of  risk  from  the  ideal.  An  abundance  of 
other  theories  of  risk  are  cited  by  Wendt  (1970)  in  his  excellent  review  of 
utility  and  risk,  and  Pollatsek  and  Tversky  (1969)  examine  the  perception  of 
risk  from  the  standpoint  of  measure  theory  to  develop  a quantitative  psycho- 
logical theory  of  risk. 

At  this  point  in  the  route- selection  research,  there  is  no  basis  upon 
which  to  choose  a particular  theory  from  among  the  many  to  predict  pilot  deci- 
sion behavior.  One  could  choose  a very  complicated  "new"  theory  or  a very 
simple  "old"  theory.  However,  in  the  absence  of  even  the  first  indication  of 
how  pilots  behave,  it  has  been  decided  that  the  choice  of  a simple  "old"  theory 
that  has  enjoyed  some  success  is  probably  the  most  prudent  approach.  Should 
the  results  of  the  research  turn  out  to  be  less  than  satisfactory,  it  would  re- 
main the  task  of  future  researchers  to  enrich  the  theory.  Edwards'  (1955) 

SEU  model  serves  as  a starting  point,  with  the  expectation-variance  principle 
being  incorporated  to  account  for  risk. 
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Since  the  SEU  model  is  to  be  used,  F(Oj)  is  defined  as  a utility  function 
and  pjj  is  a subjective  probability.  Thus,  the  expected  utility  of  alternative 
a;  is 

4 

Ua  = E Pij  F(Oj)f 

1 j-1 

the  expected  outcome  of  alternative  a^  is 

4 

Ea.  = E Pij  (Oj) 

l j=l  J J 

and  the  variance  of  a-  is 

4 

V = E Py  (Oj,2  - Ea2  . 

1 j-i  1 

According  to  the  expectation- variance  principle,  the  pilot  will  choose  a.  so  as 

to  maximize  U„  ; and,  failing  to  find  a unique  course  of  action,  will  choose  so 

i |i 

as  to  minimize  V among  those  alternatives  to  which  he  is  otherwise  indiffer- 
ent. 

Now,  one  may  intuitively  argue  that  during  route  planning,  a pilot  cannot 
force  himself  to  consider  his  own  demise.  It  is  surmised  that  investigations 
conducted  to  elicit  his  utility  function  would  find  that  he  possesses  practically 


r 


infinite  disutility  for  outcomes  which  include  his  failure  to  survive.  By  com- 
parison, he  would  possess  almost  infinite  utility  for  outcomes  that  include  his 
survival.  Also  by  comparison,  all  outcomes  that  include  his  survival  would  be 
assigned  utilities  that  are  almost  identical.  Since  utilities  are  unique  up  to  a 

positive  linear  transformation,  an  approximation  for  Va^  can  be  obtained  by 
| 1 if  j = 2,4 

F(0.)  = < . Thus 

3 I 0 if  j = 1,3 

4 

u*t  ’1  % F'°j> 

= Pi2  + Pi4  = pi  > 

where  is  simply  the  probability  of  surviving  the  mission,  regardless  of  whether 
the  target  lives  or  dies. 

From  the  above  relation  one  may  predict  that  the  pilot  will  choose  that 
alternative  which  maximizes  his  chances  for  survival,  a result  that  is  intuitively 
appealing.  However,  carried  to  the  extreme,  application  of  the  decision  rule 
would  predict  that  the  pilot  will  simply  choose  not  to  conduct  a mission  at  all 
since  considerations  of  the  chances  that  exist  for  destroying  the  target  by 
employing  a given  route  alternative  have  been  dropped  from  the  decision  rule . 

To  avoid  such  a result,  the  only  alternatives  that  will  be  allowed  are  those 
that  produce  some  positive  probability  of  destroying  the  target.  In  other  words, 
the  pilot  is  constrained  to  choose  a route  which  does  not  violate  the  requirements 
of  an  attack,  that  are  imposed  by  the  physical  limitations  of  the  ordnance  to  be 
delivered.  After  having  achieved  this  minimum  threshold  with  his  route,  sur- 
vival will  be  the  pilot's  only  consideration. 
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One  may  also  note  from  the  utility  relation  that,  faced  with  alternatives 
to  which  he  is  otherwise  indifferent,  the  pilot  will  have  to  employ  an  alternate 
rule  such  as  flipping  a coin  to  decide  which  alternative  to  choose.  Minimiza- 
tion of  variance  does  not  lead  to  a decision  since,  in  this  case, 

\ - prpi2  “ Vak  - p,  -pj2  w>“”  p,  - p, . 

Practical  Application  of  the  Decision  Rule 

Discussion  of  the  decision  rule  derived  for  the  route- planning  decision 
model  has  indicated  that  one  simply  has  to  determine  Pi  for  each  alternative  plan 
Uj  in  order  to  apply  the  decision  rule.  However,  there  are  at  least  two  reasons 
why  this  is  not  the  approach  that  has  been  followed  in  practice.  Of  most  impor- 
tance is  the  fact  that  Pj  is  a probability  of  survival  which  must  be  elicited  from 
the  decision  makers  themselves.  It  was  felt  that  it  might  be  very  difficult  to 
obtain  directly  from  a decision  maker  an  accurate  estimate  of  his  own  chances 
for  survival.  Of  secondary  importance  is  the  fact  that  for  reasons  to  be  explained, 
maximization  of  Pi  requires  that  an  exponential  criterion  function  be  employed. 
To  avoid  potential  computational  difficulties,  a linear  criterion  function  was 
desired.  Thus,  it  was  decided  to  take  an  indirect  approach  to  the  problem  of 
measurement;  that  is,  to  measure  a variable  that  may  be  related  to  survival 
probability,  but  which  is  couched  in  terms  more  familiar  to  the  pilot/decision- 
maker and  which  permits  a linear  criterion  function. 

A clue  to  the  possible  difficulty  of  measuring  survival  probabilities 
directly  was  provided  during  informal  discussions  with  pilots  in  which  they  stated 
that  they  choose  routes  so  as  to  minimize  "exposure"  to  the  enemy.  It  is  unclear 
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what  units  were  being  used  to  measure  "exposure",  and  it  is  also  unclear  what 
relationship  was  implied  between  "exposure"  and  the  variables  that  describe  a 
route  selection  situation.  However,  it  is  very  clear  that  pilots  can  talk  about 
"exposure"  and  the  threat  they  feel  from  this  exposure.  In  crisis  situations 
they  apparently  do  not  consider  survival  or  success  explicitly.  Bather,  they 
implement  intuition  and  training  in  reacting  to  a particular  situation.  Thus,  they 
arrive  at  a course  of  action  that  they  perceive  as  enhancing  their  chances  for 
survival  or  for  mission  accomplishment. 

We  have,  therefore,  resorted  to  the  use  of  a variable  that  we  choose  to 
call  "perceived  threat",  which  we  would  maintain  is  equivalent  to  the  variable 
"exposure"  alluded  to  by  the  pilot/decision-maker.  As  will  be  seen,  it  is  pos- 
sible to  view  the  evaluation  of  a route  alternative  in  terms  of  perceived  threat 
as  being  equivalent  to  evaluation  in  terms  of  survival  probability.  However, 
instead  of  maximizing  survival  probability,  the  decision  rule  becomes  minimiza- 
tion of  perceived  threat.  Moreover,  for  reasons  already  cited,  it  is  possible  to 
directly  ascertain  the  subjective  quantity  "perceived  threat"  whereas  measure- 
ment of  survival  probability  might  be  extremely  difficult. 

Now,  there  arc  many  relationships  between  perceived  threat  and  survival 
probability  that  could  be  hypothesized.  However,  it  is  not  within  the  scope  of  the 
present  research  program  to  examine  these  hypotheses  extensively.  Instead,  it 
is  simply  the  intent  that  a relationship  be  chosen  which  meets  the  requirements 
stated  earlier  and  whose  validity  could  be  tested  if  desired.  As  discussed  below, 
both  of  these  criteria  are  satisfied  by  the  relationship  that  has  been  chosen. 
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Deferring  for  the  moment  any  justification  for  the  relationship,  let 
rt  - -In  (Pj). 

Then  tj  = B • r^ 

where  B is  a constant,  Pj  is  the  probability  of  survival  as  before,  and  t^  is  the 
perceived  threat  of  route  alternative  a-.  The  decision  rule  is  to  select  that 

alternative  aj,  i = 1,  2,  ...  which  minimizes  tj. 

* 

At  least  five  interesting  observations  may  be  made  with  respect  to  the 
transformation  employed  to  obtain  t^.  First,  the  transformation  yields  logically 
consistent  results.  As  survival  probability  decreases  perceived  threat  in- 
creases. Furthermore,  perceived  threat  is  zero  when  survival  is  considered 
certain.  Second,  if  B were  known,  one  could  relate  a value  of  tj  to  the  proba- 
bility of  survival.  This  observation  suggests  that  a study  could  be  conducted  to 
test  the  hypothesized  relationship  although  no  plans  presently  exist  for  such  a 
study.  Thus,  the  desired  quality  of  testable  validity  has  been  achieved. 

Next,  the  decision  rule  has  indeed  been  converted  from  one  involving 
maximization  of  utility  to  one  involving  minimization  of  perceived  threat. 
Furthermore,  the  process  of  threat  minimization  is  equivalent  to  minimization 
of  expected  disutility. 

Another  observation  is  that  a procedure  to  be  discussed,  which  is  used 
to  represent  the  search  for  alternatives , produces  a route  that  is  actually  a set 
of  route  segments  connected  so  as  to  produce  a continuous  route.  This  imme- 
diately suggests  that  the  utility  function  Pj  can  be  expressed  as  the  product  of 
conditional  survival  probabilities  that  might  be  derived  for  each  segment. 


That  is  P.  = II  p 

1 m sj 


where  ps.  is  the  probability  of  survival  on  route  segment  j and  R is  the  route 


under  investigation.  An  algorithm  employing  Pj  as  the  function  to  be  maxi- 
mized would  then  have  a product  as  the  criterion  as  stated  earlier.  However, 
through  the  transformation  used  to  obtain  t j,  the  problem  has  been  converted 


into  one  in  which  the  criterion  is  a sum.  That  is,  assuming  tg.  = -Bln(pg  ),  then 


*i  ^ ts 
j€R  j 


where  tg^  is  the  perceived  threat  for  a route  segment  and  corresponds  to  pg  , 


and  B is  the  constant  defined  previously.  Thus,  the  transformation  yields  the 
desired  linearity. 


The  final  observation  is  that  the  use  of  a sum  as  the  criterion  with  t s . 


as  a component  of  the  sum  is  similar  to  the  procedure  employed  by  Clark  (1966) 
in  his  model  of  ground  route  selection.  Clark  referred  to  the  terms  of  the  sum 
as  "tactical  difficulties"  and  spoke  of  them  as  elements  of  an  additive  utility 
system.  Unfortunately,  development  of  Clark's  model  is  incomplete  (see 
page  12 ). 

Exclusive  of  the  appealing  properties  discussed  above,  some  justification 


for  the  choice  of  the  function  relating  tj  and  Pj  (or  ts  and  ps  ) lies  in  the  fact 

j J 


that  under  certain  assumptions,  the  relation  can  be  derived  by  applying  principles 
that  have  served  as  a basis  for  many  developments  in  classical  psychophysics. 
That  is,  there  may  be  historical  justification  for  the  chosen  form  of  the  relation- 
ship. 
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First  of  all,  the  underlying  problem  is  actually  a classical  problem  in 
psychophysics.  There  is  measurable  environmental  information  available  in 
the  form  of  various  stimuli  which  relate  to  the  uncertainty  of  survival.  How- 
ever, the  pilots  apparently  react  to  a psychological  sensation  that  they  refer  to 
as  threat  or  exposure.  The  problem  is  to  relate  the  environmental  stimuli  to  the 
psychological  sensation  called  threat. 

The  development  starts  with  the  "fundamental  formula"  (Rand,  1912) 
of  classical  psychophysics  called  Weber's  Law: 

AS  = C 
S 

where  S is  a physical  measure  of  stimulus  magnitude,  AS  is  a small  incremental 
change  in  S,  and  C is  a constant  difference  in  sensation  resulting  from  the  relative 
stimulus  change.  Based  on  classical  theory,  any  function  relating  sensation  to 
stimulus  magnitude  must  satisfy  Weber's  Law. 

Now,  if  the  very  critical  assumption  is  made  that  the  probability  of 
survival  Pj  is  in  actuality  a stimulus  and  if  we  let  ¥ be  a psychological  sensation, 
then  Weber's  Law  may  be  restated  as 

A¥  = KAPj 
Pi 

where  A'?  is  the  sensation  increment  resulting  from  AP^  and  K is  a constant  of 
proportionality.  The  assumption  that  Pj  is  a stimulus  is  equivalent  to  the  assump- 
tion that  all  the  environmental  stimuli  relating  to  the  uncertainty  of  survival  have 
in  some  way  been  combined  to  produce  Pj  , and  just  tins  one  stimulus  is  presented 


nr, 


to  the  pilot. 


Viewing  the  formula  for  AP^  as  a differential  equation,  integration  yields 


¥ = K In  Pi  + D 

where  D is  a constant  of  integration.  To  evaluate  D,  one  introduces  the  notion 
of  threshold;  i.e. , that  ¥ vanishes  with  some  liminal  value  of  Pj  called  p.  Then, 
D = -K  In  p.  Therefore, 

¥ = K (In  P - In  p) 
or 

Y = K In  (P^p). 

Since  the  scale  of  measurement  is  arbitrary,  one  may  specify  the  liminal  value 
of  Pi  as  p = 1.  This  yields  the  final  result 
¥ = K In  (Pj) 

which  relates  a stimulus  to  a sensation.  In  general,  this  relation  is  known  as 
Fechner's  Law.  It  can  be  seen  that  it  is  identical  to  the  relation  for  t.  stated 
earlier  if  K is  taken  as  -B.  Also,  the  constant  K (or  B)  may  be  experimentally 
determined.  Thus,  we  have  established  the  relation  that  was  desired. 

Now,  a critical  assumption  in  the  above  development  may  be  stated  as 
follows : 

Equally  often  noticed  differences  are  equal,  unless  always  noticed  or 
never  noticed.  (See  Luce  and  Edwards,  1958.) 

This  assumption  means  that  all  "just  noticeable"  threat  situation  differences 

(AP-)  represent  constant  differences  in  sensation.  Had  the  assumption  been 

) 
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that  the  just  noticeable  differences  represent  constant  ratios  in  sensation;  i.e. , 

AY  _ KAPj 
Y P. 

l 

then  the  result  would  have  been  much  different.  Such  an  assumption  is  made  in 
the  psychophysical  theory  developed  by  S.  S.  Stevens 3. 

Representing  the  Search  for  Alternatives 

One  of  the  most  challenging  aspects  of  developing  a model  to  describe 
the  route  planning  task  is  the  derivation  of  a method  to  predict  the  results  of 
the  process  by  which  a pilot  isolates  candidate  route  alternatives  in  the  first 
place.  Attention  is  now  directed  to  this  task. 

In  accordance  with  pilot  comments,  the  hypothesis  is  constructed  that  a 
helicopter  pilot  is  mainly  guided  in  his  selection  of  a route  alternative  by  his 
interpretation  of  intervisibility  relationships  that  exist  throughout  the  battlefield. 

In  his  search  for  alternative  approaches  to  a target,  the  pilot  first  visualizes 
areas  on  the  battlefield  that  are  masked  from  the  view  of  known  or  suspected  enemy 
weapons.  From  this  set  of  masked  areas  he  is  able  to  choose  a series  of  areas 
whose  proximity  to  one  another  is  such  that  the  series  can  be  connected  to  form 
an  avenue  of  approach.  Once  this  avenue  is  determined,  the  route  alternative  can 
be  finalized  by  selecting  a series  of  route  segments  that  follow  the  general  shape 
of  the  approach  corridor. 

3 For  more  discussion  of  the  mathematical  developments  in  the  lasf  few 
paragraphs,  see  Miller  (1964). 
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In  the  vicinity  of  the  target,  the  distribution  of  intervisibility  areas  is 
still  important,  since  the  pilot  desires  to  conceal  the  details  of  his  attack  plan 
until  the  very  last  moment.  Thus,  he  selects  a route  alternative  leading  to  his 
firing  point  from  among  the  masked  areas  in  the  vicinity  of  the  target.  Just 

4 

prior  to  initiation  of  the  firing  sequence  he  climbs  above  the  knap-of-the-earth 
flight  profile  to  achieve  intervisibility  and  subsequently  to  conduct  his  attack. 

Once  the  attack  has  been  completed,  he  again  selects  a route  that  has 
characteristics  similar  to  his  approach  route.  In  fact,  the  masked  areas  that 
formed  his  approach  corridor  may  serve  as  his  exit  corridor.  Protected  route 
segments  during  the  escape  phase  of  the  mission  are  as  desired  as  were  those 
selected  during  earlier  portions  of  the  mission. 

The  above  conceptualization  of  the  search  for  alternatives  serves  as 
inspiration  for  the  procedure  that  has  been  adopted  to  represent  the  process  in 
the  descriptive  route  selection  model.  The  procedure  is  an  improvement  over 
that  offered  by  Clark  (1966)  in  that  it  explicitly  portrays  the  existence  of  areas 
over  the  battlefield  through  which  the  search  for  alternatives  is  more  likely  to 
be  concentrated.  It  is  also  more  readily  comparable  to  the  process  that  may 
actually  be  used  by  the  decision  maker. 

^ Knap-of-the-earth  flight  is  flight  conducted  at  very  low  and  almost  con- 
stant altitude  above  the  local  terrain.  Over  wooded  areas,  sufficient  elevation 
is  used  to  clear  the  most  prominent  tree  tops.  Over  open  terrain,  flight  may 
be  conducted  at  elevations  below  the  height  of  trees  in  adjacent  woods  or  tree 
lines . 
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Our  procedure  visualizes  the  boundaries  of  the  various  intervisibility 
areas  as  providing  terminal  positions  for  major  segments  of  a route  alterna- 
tive. A major  segment  of  a route  alternative  is  defined  to  be  one  in  which  the 
* 

intervisibility  relationship  that  exists  with  all  enemy  weapons  does  not  change. 
Thus,  by  definition,  a major  route  segment  extends  from  one  boundary  of  an 
intervisibility  area  to  another.  Within  a major  segment,  several  minor  seg- 
ments may  exist.  These  segments  are  differentiated  from  one  another  merely 
by  changes  in  direction.  They  are  included  to  permit  shaping  of  the  route  alter- 
native within  an  intervisibility  area  and  result  in  a route  that  may  follow  the 
general  shape  of  the  intervisibility  area. 

To  illustrate  the  procedure  adopted  to  permit  such  a representation,  we 
present  the  following  example.  Consider  the  line-of-sight  map"*  of  Figure  1. 
This  map  has  been  prepared  for  enemy  weapons  located  at  the  positions  marked 
(1)  and  (2)  with  the  shaded  areas  being  areas  of  intervisibility.  Moreover,  the 
route  planning  task  is  for  an  attack  upon  the  weapon  located  at  position  (2). 

Now,  Figure  2 shows  a simplification  of  the  map  of  Figure  1.  The  vari- 
ous areas  have  been  stylized  with  straight  line  segment  boundaries.  Moreover, 
each  area  has  been  coded  to  specify  the  intervisibility  condition  existing  within 

5 

A line-of-sight  map  is  a map  prepared  to  show  areas  of  a battlefield 
which  are  intervisible  with  the  enemy  weapons  on  the  battlefield,  given  that  the 
helicopter  is  Tying  a knap-of-the -earth  flight  profile.  A procedure  to  prepare 
such  a map  is  described  in  Appendix  A. 


its  boundaries.  The  coding  is  as  follows: 

0 if  a line  of  sight  exists  to  neither  weapon 

1 if  a line  of  sight  exists  to  the  target  weapon  only 

2 if  a line  of  sight  exists  to  the  supporting  weapon  only 

3 if  a line  of  sight  exists  to  both  weapons. 

The  boundaries  shown  in  Figure  2 are  those  that  provide  terminal  points  for 
the  major  route  segments  discussed  earlier. 

To  permit  selection  of  minor  route  segments,  the  irregularly  shaped 
areas  of  Figure  2 are  broken  into  sets  of  connected  convex  areas  still  having 
straight  line  segment  boundaries  as  shown  in  Figure  3.  The  minor  route  seg- 
ments will  have  their  terminal  points  on  the  set  of  boundaries  added  to  obtain 
Figure  3 from  Figure  2.  One  will  note  that  the  same  coding  for  intervisibility 
has  been  maintained  and  the  areas  have  been  numbered  for  reference  purposes. 
Moreover,  the  line  segments  forming  area  boundaries  have  also  been  numbered. 
The  number  of  line  segments  is  designated  as  IMAX  and  in  this  case  IMAX  = 34. 

Now,  each  minor  route  segment  is  represented  as  a straight  line.  Its 
terminal  positions  are  on  the  area  boundaries  as  discussed  previously,  and  a 
series  of  these  minor  segments  connected  properly  forms  a major  route  seg- 
ment. The  convexity  of  areas  in  Figure  3 is  required  to  permit  straight  lines 
to  be  used  to  form  minor  route  segments. 

In  general,  a minor  route  segment  could  join  any  two  points  on  the  bound- 
aries of  one  of  the  convex  areas.  However,  we  have  found  it  necessary  to  dis- 
cretize the  process  in  order  to  reduce  the  number  of  alternatives  to  a compu- 
table numbe  r . It  should  lx'  remembered  that  we  are  merely  trying  to  represent 
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Figure  3. — Convex  Area  Representation 


the  process  by  which  a search  for  alternatives  is  conducted.  The  decision- 
maker may  in  fact  consider  either  a greater  number  of  alternatives  or  a smaller 
number.  We  are  constrained,  however,  by  definite  computer  limitations. 

In  the  procedure  a minor  route  segment  extends  from  one  "neighbor"  to 
another.  Neighbors  are  defined  as  points  obtained  by  dividing  each  boundary 
line  segment  into  intervals  of  approximately  equal  length.  Thus,  if  one  end  of  a 
minor  route  segment  is  at  point  J on  line  I,  then  the  other  end  is  at  a neighbor  of 
I,  J;  that  is,  it  is  at  point  N on  line  M where  line  M is  any  other  line  that  can  be 
reached  from  line  I without  crossing  an  intervening  line.  The  points  on  a line  are 
numbered  sequentially  from  one  end  of  a boundary  segment  to  the  other  end.  As 
implied  in  Figure  3,  the  order  of  numbering  of  the  boundary  line  segments  is  of 
no  importance,  other  than  that  each  integer  between  1 and  IMAX  should  be  used. 

The  procedure  used  to  divide  the  boundary  line  segments  into  intervals 
is  designed  to  yield  approximately  equal  spacing  between  neighbors  around  the 
entire  boundary  of  an  area.  If  DELP  is  specified  as  the  desired  spacing  between 
points,  and  RD  is  the  length  of  a boundary  segment,  then  the  procedure  defines 
neighboring  points  on  a segment  according  to  the  following  rules : 

a.  the  distance  from  either  end  of  the  segment  to  the  first  point 
on  the  segment  is  never  greater  than  0.9  • DELP,  and 

b.  if  more  than  one  point  exists  on  a segment,  the  points  are 
equally  spaced  at  intervals  of  DELP,  with  the  first  and  last 
points  being  equally  spaced  from  the  respective  line  segment 
end  points. 

As  may  be  seen  in  Figure  4,  the  procedure  yields  fairly  equal  overall  spacing 


1 


between  points. 
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Figure  4. — Neighbors 


So  far,  nothing  has  been  said  about  the  representation  of  an  attack  upon 
the  target.  A scheme  must  exist  to  permit  a transition  from  the  approach 
route  into  a firing  position.  The  firing  activity  and  then  a transition  back  into 
the  escape  phase  of  the  route  must  also  be  represented.  The  representation  de- 
scribed so  far  is  valid  only  for  the  enroute  movement  phases  of  the  mission. 

To  solve  the  dilemma,  concepts  described  so  far  are  merely  extended  to 

cover  the  attack  phase.  A series  of  equally  spaced  rays  extending  outward  from 
the  target  is  visualized.  In  general,  there  are  NR  of  these  rays  and  they  serve 

to  define  the  directions  from  which  attacks  can  be  conducted.  Along  each  ray 

are  distributed  a series  of  points  at  which  a route  could  intersect  the  attack 

ray.  Associated  with  each  of  these  points  is  another  point  at  which  an  attack 

can  be  broken  at  termination  of  fire  (upon  impact  of  the  missile  that  has  been 

launched) . From  this  latter  point  a transition  route  segment  can  be  selected 

to  permit  movement  into  the  mission  escape  phase. 

The  procedure  used  to  select  points  on  each  attack  ray  is  structured  to 

place  the  first  entry  point  at  a range  of  RMAXL  + VHEL  • TDELAY 

where  RMAXL  is  the  maximum  range  at  which  a missile  can  be 

launched  at  the  target, 

VHEL  is  the  forward  speed  of  the  helicopter  that  will  be 
employed  during  the  attack,  and 

TDE  LAY  is  the  time  required  to  aim  and  launch  a missile 
once  the  attack  position  has  been  reached. 

It  is  assumed  that  a time  delay  of  TDELAY  will  cccur  after  the  entry  point  is 

reached  so  that  the  launch  associated  with  this  entry  point  will  occur  at  a range 
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of  RMAXL  from  the  target.  However,  if  this  entry  point  is  not  within  the 
boundaries  of  the  battlefield,  the  range  is  shortened  to  place  the  entry  point  at 
the  battlefield  boundary. 

After  the  first  entry  point  on  a ray  has  been  selected,  then  succeeding 
points  are  placed  on  the  ray  at  intervals  of  DE  LR  (equivalent  to  DE  LP , but  asso- 
ciated with  attack  rays)  until  one  of  two  constraints  terminate  the  procedure.  The 
first  constraint  is  that  the  launch  range  associated  with  a given  entry  point  range 
must  be  greater  than  the  minimum  effective  missile  launch  range  RMINL.  If 
R is  the  entry  range,  then 

RLNCH  = R - VHEL  • TDELAY 
is  the  launch  range. 

The  other  constraint  is  that  the  exit  point  associated  with  a given  entry 
point  must  lie  outside  the  minimum  approach  range  to  the  target  RMAPP.  The 
exit  point  is  the  position  the  helicopter  would  occupy  at  the  time  of  missile 
impact,  given  that  the  missile  was  launched  at  the  launch  range  RLNCH  specified 
earlier.  The  approach  range  is 

RAPP  = RLNCH  • (1  - VHEL/VMIS) 
where  VMB  is  the  average  flight  velocity  of  the  missile. 

The  reader  should  note  that  two  different  types  of  fire  or  modes  of  attack 
may  be  represented  by  the  above  procedure.  Running  fire  (RF)  is  a mode  of 
attack  in  which  the  helicopter  continues  to  close  on  the  target  after  a missile 
is  launched  (VHEL  > 0).  The  firing  run  does  not  terminate  until  the  missile 
impacts.  Hover  fire  (II K)  is  a mode  of  attack  in  which  the  forward  speed  is 
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zero  (VHEL  = 0)  during  the  flight  of  the  missile.  The  helicopter  essentially 


"pops-up"  to  a firing  position,  maintains  the  position  while  the  missile  is  in 
flight  and  then  "pops  down"  to  commence  the  escape  phase  of  the  mission.  In 
hover  fire,  the  entry  point,  the  launch  point  and  the  exit  point  are  coincident. 

The  reader  should  also  note  that  it  is  implied  that  the  three  points  de- 
fining an  attack  (entry,  launch,  exit)  lie  on  a straight  line  that  emanates  from 
the  target  and  which  is  elevated  sufficiently  to  provide  a line  of  sight  to  the  tar- 
get from  the  entry  point.  The  line  represents  the  flight  profile  of  the  missile, 
and  the  implied  relationship  satisfies  the  requirement  that  the  helicopter  re- 
main intervisible  with  the  target  in  order  that  the  missile  might  be  guided  to 
the  target.  As  a simplification,  it  is  assumed  that  the  helicopter  is  inter- 
visible with  all  enemy  weapons  on  the  battlefield  during  the  entire  attack.  This 
includes  transition  to  and  from  the  firing  run  and  the  firing  run  itself.  More 
will  be  said  about  this  assumption  in  Chapter  4 and  Chapter  5. 

A completed  picture  of  the  final  results  of  the  procedure  described  so 
far  is  presented  in  Figure  5.  To  the  neighbors  shown  in  Figure  4 have  been 
added  the  attack  rays  showing  attack  entry  points  (L)  and  attack  exit  points  (E). 
A search  for  alternatives  is  conducted  in  this  environment. 

Now  that  we  have  described  all  the  procedures  used  to  specify  the  en- 
vironment in  which  a search  for  alternatives  is  conducted,  we  may  present  the 
definitions  of  several  different  quantities  that  are  required  in  the  route  selec- 
tion procedure  to  be  described.  These  quantities  are  all  calculated  as  part  of 

is 
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the  data  preparation  effort  that  must  be  accomplished  before  the  route  selection 
procedure  can  be  applied  to  a given  tactical  scenario. 

As  previously  defined,  let  IMAX  be  the  total  number  of  line  segments 
used  to  specify  the  boundaries  of  the  knap-of-the-earth  intervisibility  areas, 
and  let  NR  be  the  number  of  attack  rays  considered.  Define  IM  as  the  number 
of  line  segments  and  attack  rays;  i.  e.  , IM  = IMAX  + NR.  Then, 


NPT(I)  = the  number  of  neighbor  points  on  each  line  segment 

for  I = 1,  ...  IMAX;  or,  the  number  of  entry  points 
on  each  attack  ray  for  1 = IMAX  + 1 , ...  IM. 

position  coordinates  of  each  point  J on  line  segment 
I for  1 = 1,  . . . , IMAX:  or,  position  coordinates  of 
each  exit  point  J on  attack  ray  I for  I = IMAX  + 1, . . . IM; 
where  J = 1,  . . . , NPT(I)  for  all  I. 

XENTRY(I,  J)  = position  coordinates  of  each  entry  point  J on  attack 
YENTRY(I,  J)  ray  I where  1 = 1,  . . . , NR  and  J = 1,  ...  NPT(I  + IMAX) 


XLOCH(I,  J) 
YLOCII(I,  J) 


JMAX  = the  maximum  number  of  points  appearing  on  any  line 

segment  or  attack  ray 
= max$NPT(I)} 

1 - I s IM 

J2MAX  = the  maximum  number  of  entry  points  appearing  on  any 
attack  ray;  i.  e.  , 
max  {NPT(I)} 

IMAX  + 1 — I — IM 

I 

Finally,  if  NL  is  the  total  number  of  areas  formed  by  the  IMAX  line 
segments,  then 


LOS(K)  = 


0 if  a LOS  exists  to  neither  weapon  from  area  K, 

1 if  a LOS  exists  to  the  target  weapon  only  from  area  K , 

J 2 if  a LOS  exists  to  the  supporting  weapon  only  from  areaK, 
3 if  a LOS  exists  to  both  weapons  for  area  K; 

K = 1,  NL. 
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This  coding  is  consistent  with  that  presented  earlier  in  the  discussion  of  Figures 
2 and  3.^ 

Several  other  quantities,  not  discussed  so  far,  are  required  to  facilitate 
the  association  of  neighbor  points  and  attack  ray  points  with  the  intervisibility 
areas.  No  method  of  association  has  been  given  so  far.  First,  define  the  coor- 
dinates of  one  end  of  a given  line  segment  I as  XL(I),  YL(I)  where  I - 1, . . . , IMAX. 
Define  the  coordinates  of  the  other  end  as  XR(I) , YR(I).  Then 

NFL(I)  = the  number  of  the  intervisibility  area  that  lies  to  the  left 
of  the  line  segment  with  origin  at  XL(I) , YL(I) 

NFR(I)  = the  number  of  the  intervisibility  area  that  lies  to  the  right 
of  the  line  segment  with  origin  at  XL(I) , YL(I). 

Note  that  the  intervisibility  conditions  that  exist  on  either  side  of  line  segment  I 
can  be  determined  by  first  finding  LI  = NFL(I)  and  L2  = NFR(I)  and  by  then  find- 
ing LOS(Ll)  and  LOS(L2).  Note  also  that  the  same  conditions  exist  for  each 
neighbor  point  J on  line  segment  I. 

Following  a similar  procedure  we  define  the  variables  NFRL(I,J), 

NFRR(I, J) , MFRL(I,J)  and  MFRR(I,J).  The  first  two  of  these  variables  specify 
the  intervisibility  areas  that  lie  to  the  "left"  and  "right",  respectively,  of 
entry  point  J on  attack  ray  I.  The  second  two  variables  are  associated  in  an 


6 The  research  has  never  considered  more  than  two  enemy  weapons  simul- 
taneously, so  the  coding  for  LOS(K)  is  general  enough  to  meet  the  needs  of  the 
present  research.  In  more  general  studies  involving  n weapons,  a coding  scheme 
permitting  n [nl  distinct  possibilities  would  be  required. 
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identical  way  with  exit  points.  Now,  since  we  are  dealing  with  points  and  not 
lines,  the  idea  of  direction  from  a point  is  really  not  applicable.  However,  for 
reasons  to  be  explained,  provisions  are  made  for  two  intervisibility  areas  adja- 
cent to  each  entry  point  and  to  each  exit  point.  The  "left"  and  "right"  associa- 
tion is  arbitrary.  In  fact  the  two  areas  may  be  the  same.  To  facilitate  under- 
standing of  this  discussion  the  reader  should  consult  Figure  5. 

Within  the  small  area  that  is  circled  in  Figure  5,  note  that  one  of  the 
attack  entry  points  (L)  lies  extremely  close  to  one  of  the  intervisibility  bound- 
ary line  segments.  In  an  approximation  it  seems  unduly  restrictive  to  consider 
the  point  as  truly  lying  within  just  one  area.  If  this  procedure  were  adopted, 
then  a route  using  L and  approaching  from  the  right  would  first  have  to  pass 
through  a neighbor  on  the  boundary  line  segment  adjacent  to  L.  A more  realis- 
tic route  would  be  one  in  which  L is  approached  directly,  bypassing  the  line 
segment  neighbor.  Therefore,  we  allow  each  entry  point  and  each  exit  point 
to  be  associated  with  as  many  as  two  intervisibility  areas.  In  practice  we  have 
used  a fifty  meter  distance  from  a line  segment  to  differentiate  whether  a point 
is  considered  to  be  only  within  a single  intervisibility  area  or  to  be  lying  on  the 
area  boundary.  We  have  found  in  practice  that  a better  route  representation 
might  have  resulted  had  a greater  separation  been  allowed.  Also,  an  improve- 
ment might  have  been  realized  if  more  than  two  areas  had  been  allowed.  In 
some  cases  a point  is  extremely  close  to  the  intersection  of  three  or  more 
boundary  line  segments. 
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The  Algorithm 


A route  is  specified  as  starting  at  point  JSTRT  on  intervisibility  boun- 
dary line  segment  ISTRT . The  route  must  terminate  on  one  of  the  boundary 

segments  NBOUND(I)  where  1=1 NBD.  Furthermore, the  route  must 

include  one  and  only  one  attack.  That  is,  exactly  one  of  the  route  segments 
must  pass  from  an  attack  ray  entry  point  to  its  associated  exit  point.  This 
fact  is  indicated  if  one  of  the  route  segment  end  points  is  a point  I,  J where 
I > IMAX.  (The  route  selection  procedure  assumes  attack  rays  numbered 

IMAX  + 1 IMAX  + NR,  and  intervisibility  boundaries  numbered 

1,  ...,  IMAX.) 

Now,  attention  is  directed  to  the  procedure  that  has  been  developed  to 
select  a route  having  the  characteristics  specified  above.  One  may  note  that 
there  are  a large  number  of  alternative  routes  that  could  be  structured  to  have 
the  desired  characteristics  and  which  are  composed  of  route  segments  defined 
according  to  the  rules  of  the  preceding  section.  The  decision  rule  defined 
previously  is  used  to  identify  that  route  alternative  which  should  be  selected. 
That  is,  we  select  the  alternative  a associated  with 
U = min  jt.  ^ 


w Ik  re  t.  is  the  perceived  threat  of  route  alternative  a^,  and  the  set  a = ja^ 
contains  ill  alternatives  isolated  by  the  alternative  search  procedure. 

The  reader  should  note  that  a route  alternative  a.  actually  consists  of 
N route  segments  ( s . ^ where  j = 1,  2 N.  Then,  for  alternative  a. 

i -jj 


r>3 


where  tSj  is  the  perceived  threat  associated  with  segment  j.  This  formulation  is 
the  result  of  the  derivation  of  tj  from  the  subjective  probability  of  survival  Pj. 
However,  the  formulation  also  implies  that  we  are  assuming  that  the  segments 
are  utility  independent  and  the  utility  of  a route  is  a simple  sum  of  the  segment 
disutilities  (Fishburn  and  Keeney,  1975),  which  we  are.  Justification  of  the 
assumption  lies  in  the  relation  that  is  proposed  to  exist  between  Pj  and  tj  and 
the  intuitive  beliefs  that:  (1)  a pilot  would  consider  his  chances  for  survival 
along  one  segment  to  be  independent  of  those  along  another  segment,  and  (2)  an 
increase  in  perceived  threat  on  one  segment  will  always  increase  the  route  per- 
ceived threat.  However  our  intuitive  beliefs  remain  to  be  tested. 

Now,  one  method  of  selecting  a route  would  be  to  evaluate  the  threat  repre- 
sented by  each  route  segment,  and  by  enumeration,  determine  that  route  having 
the  least  threat.  Such  a procedure  is  completely  infeasible  for  any  but  the  simplest 
tactical  situation  representations.  By  far,  a more  efficient  procedure  is  a dynamic 
programming  technique  first  suggested  by  Jefferis  and  Fegley  (1965)  for  routing 
problems  and  later  modified  by  Clark  (1966)  to  solve  the  armored  vehicle  route 
selection  problem. 

According  to  the  procedure,  optimal  paths  to  various  points  in  the  matrix 
of  neighbor  points  are  recursively  identified  until  a route  path  having  all  the  re- 
quired characteristics  is  identified.  As  each  point  in  an  optimal  route  is  determined, 
the  previous  point  in  the  route  is  identified,  so  that  it  is  always  possible  to  trace 

backward  through  a route  at  any  stage  to  find  a complete  description  of  the  route. 
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The  dynamic  programming  algorithm  is  implemented  through  the  con- 
struction of  four  sets  U,  V,  S and  T.  The  set  U contains  all  points  to  which 
a route  having  minimum  threat  has  been  determined,  given  that  the  route 
does  not  contain  an  attack  segment.  Set  S is  similar  to  set  U but  it  contains 
only  those  points  to  which  a route  having  minimum  threat  has  been  determined, 
given  that  the  route  does  include  an  attack  segment.  All  the  entry  points  to 
members  of  set  U are  also  members  of  set  U.  Likewise,  all  the  entry  points 
to  members  of  set  S are  also  members  of  set  S.  This  differentiation  between 
sets  U and  S allows  an  escape  route  to  retrace  an  approach  route  if  necessary. 
Also,  by  convention,  an  attack  entry  point  can  appear  only  in  set  U since  at  the 
time  an  attack  entry  point  is  added  to  a route  set,  no  attack  segment  is  included 
in  the  route.  However,  the  exit  point  associated  with  the  attack  entry  point  is 
added  to  set  S since  an  attack  segment  is  included  in  that  route.  Since  the  entry 
points  to  members  of  set  S are  also  in  set  S,  the  attack  entry  point  is  both  a 
member  of  set  U and  of  set  S.  With  this  one  exception  sets  U and  S are  dis- 
joint. The  convention  is  required  to  permit  the  construction  of  a complete  route. 

So  far,  we  have  discussed  sets  U and  S which  contain  points  to  which 
optimal  paths  have  been  determined.  The  sets  V and  T contain  points  to  which 
paths  have  been  determined,  but  which  have  not  been  determined  to  be  optimal. 
That  is , paths  to  points  in  V and  T are  candidate  optimal  paths . If  a path  to  a 
point  in  V is  determined  to  be  optimal,  then  the  point  is  placed  in  set  U,  and 
since  the  path  is  known  to  be  optimal,  it  is  removed  from  set  V.  A similar 
relationship  exists  between  sets  S and  T. 
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All  neighbor  points  of  each  point  in  set  U are  contained  in  set  V with  the 
exception  of  neighbor  points  of  an  attack  entry  point  as  explained  in  a preceding 
paragraph.  However,  if  a point  that  would  normally  appear  in  set  V is  also 
included  in  set  U,  it  is  deleted  from  set  V since  it  is  known  that  the  path  to  that 
point  is  already  optimal.  One  would  not  consider  going  from  a member  of  set 
U to  a neighbor  for  which  an  optimal  route  has  already  been  determined.  By 
definition,  this  new  route  to  the  neighbor  would  not  be  as  good  as  that  which 
had  already  been  determined  for  the  neighbor. 

As  stated,  the  set  T is  similar  to  set  V as  it  contains  the  neighbors  of 
points  in  set  S.  Again,  a disjoint  relationship  is  maintained  between  S and  T 
for  the  same  reasons  cited  above. 

For  each  point  in  U , V , S and  T , the  total  threat  posed  by  the  route 
taken  to  reach  the  point  from  the  start  point  is  recorded.  Also,  the  entry  point 
of  the  last  segment  leading  to  the  member  point  is  recorded.  The  following 
symbols  may  be  used  to  clarify  the  discussion.  Define: 


M,NCI,  J 


M 


»N°  I,  < 


= relative  threat  encountered  in  reaching  (I,  J)  from 
the  starting  position  (ISTRT,  JSTRT),  given  that 
the  route  does  not  include  an  attack  segment  and 
that  the  entry  point  (M,  N)  appears  in  set  U. 

= relative  threat  encountered  in  reaching  (I,  J)  from 
the  starting  position  (ESTRT,  JSTRT),  given  that  the 
route  does  include  an  attack  segment  and  that  the 
entry  point  (M,N)  appears  in  set  S. 


5G 


w 


I 

I 


C* 

M,  N I,J 


M, 


DUMN 


= least  threat  obtainable  in  reaching  (I,  J)  from  the 
starting  position,  given  that  the  route  does  not  in- 
clude an  attack  segment  and  that  both  point  (I,  J) 
and  (M,  N)  are  members  of  set  U. 

= least  threat  obtainable  in  reaching  (I,  J)  from  the 
starting  position,  given  that  the  route  does  include 
an  attack  segment  and  that  both  point  (I,  J)  and  (M,  N) 
are  members  of  set  S.  (The  point  (M,  N)  may  be  a 
member  of  both  set  S and  set  U if  (M,  N)  is  an  attack 
entry  point. ) 

= threat  encountered  on  the  route  segment  from  point 
(M,  N)  to  point  (I,  J). 


Then 


M,NCI,  J 


C* 

A,  B M,  N 


+ DUMN 


and 


. D*  v + DUMN 
A,  B M,  N 


where  (A,  B)  is  the  entry  point  to  (M,  N)  and  is  not  equal  to  (M,  N)  or  (I,  J). 

At  a given  stage  of  the  dynamic  programming  algorithm,  the  last  point 
(M,  N)  placed  in  set  U or  set  S is  selected  as  the  point  to  be  analyzed  during 
the  stage.  First,  (M,  N)  is  considered  as  the  entry  point  to  each  of  its  neigh- 


bors (I , J) , and  the  quantities  C or  D are  computed.  Then,  the 

M,N  I,J  M,N  I,J 

points  (I,  J)  are  entered  either  in  set  V or  set  T,  and  the  point  (M,  N)  is  recorded 
as  the  entry  point  for  each  of  these  new  member  points . Finally , a search  is 


conducted  to  determine  a point  (P , R)  to  be  placed  either  in  set  U or  set  S. 
The  point  (P,  R)  is  removed  either  from  set  V or  set  T,  it  becomes  the  last 
entry  in  set  U or  set  S,  and  the  next  stage  of  the  procedure  commences . 


I 

L.  » l 
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To  determine  (P,  R),  the  quantities 


VMIN  = min 

(l,j)  e v 


f A,BCI,j} 


TMIN  = min 
(I,J)  G T 


f D 1 
CA,B  I,Jj 


are  determined  first,  where  (A,B)  are  entry  points  associated  with  the  points 
(I,  J)  and,  by  definition,  appear  in  set  U or  set  V.  Then,  if  VMIN  < TMIN, 

(P,  R)  becomes  the  point  (I,  J)  yielding  VMIN.  Otherwise  (P,  R)  is  the  point 

(I,  J)  associated  with  TMIN.  In  the  first  case, 

C*  = VMIN 

E,F  P,R 

and  in  the  second 

D*  = TMIN 

E,F  P,R 

where  (E , F)  is  the  entry  point  to  (P,  R). 

The  computer  procedure  to  be  described  below  uses  V(I,J)  to  represent 

M N^I  J*  N)  contained  in  the  arrays  IV  (I , J)  and  JV(I,J). 

likewise,  T(I,J)  represents  D , with  IT(I,J)  and  JT(I,J)  containing  the 

M,N  I,J 

subscripts  (M,N).  The  arrays  U(I,J)  and  S(I,J)  contain  the  quantities  C* 

M ,N  I,J 

and  D*  , respectively. 

M,N  I,J 

The  steps  in  the  computational  process  used  to  determine  an  optimal 
route  are  as  follows: 

1.  Initialize  the  lists  U,V,S,T,IV,JV,IT  and  JT  to  zero. 


I 


I 

I 


2.  Initialize  the  route  at  the  start  point;  i.e. , set  I = ISTRT  and 

J = JSTRT.  Also,  record  the  fact  that  the  optimal  route  to  the 
initial  point  has  been  determined  by  setting  U(I,J)  to  a small 
positive  value  (e.g. , 10-t>).  Record  the  fact  that  the  route 
to  I,  J does  not  contain  an  attack  segment  by  setting 
ESCAPE  = .FALSE. 

3.  Determine  the  threat  of  the  optimal  route  to  the  point  (I,J) 
being  analyzed;  i.e. , set 

J U(I,  J)  if  ESCAPE  = . FALSE . 

U |S(I,J)  if  ESCAPE  = .TRUE. 

n 

4.  Determine  the  next  neighbor  point  (M,N)  to  (I,J).  If  all 
neighbor  points  have  been  analyzed,  go  to  step  13.  Other- 
wise, continue. 

5.  If  the  route  to  (I,J)  includes  an  attack  segment;  i.e. , if 
ESCAPE  = .TRUE. , go  to  step  12;  otherwise,  continue. 

6.  If  an  optimal  route  to  (M,N)  has  already  been  determined; 
i.e. , if  U(M,N)  > 0,  go  to  step  4. 

7.  Determine  the  threat  to  be  encountered  in  proceeding  along 
the  route  segment  between  (I,J)  and  (M,N);  i.e. , determine 
DIJMN. 

8.  Compute  the  threat  for  a route  to  (M,N)  that  passes  through 
(I , J) ; i.e. , compute  D = DU  + DIJMN. 

9.  If  a route  to  (M,N)  has  already  been  determined;  i.e.,  if 
V(M,N)  > 0,  go  to  step  11;  otherwise,  continue. 

10.  Record  the  threat  posed  by  the  route  to  (M,N)  that  passes 
through  (I,J);  i.e. , set  V(M,N)  = D.  Also  record  the  entry 
point  to  (M,N);  i.e. , set  IV(M,N)  = I and  JV(M,N)  = J. 

Then  go  to  step  4. 


The  process  used  to  successively  determine  neighbor  points  of  (I , J)  is 
discussed  immediately  following  this  description  of  the  route  selection  compu- 
tational procedure. 
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11.  If  a route  passing  through  (I,  J)  to  (M,  N)  would  be  better  than 
the  one  already  recorded  for  (M,  N);  i.  e.  , if  D < V(M,N),  go  to 
step  10;  otherwise,  go  to  step  4. 

12.  Perform  steps  6 through  11  for  a route  that  is  known  to  contain 
an  attack  segment.  Substitute  S(M,N)  for  U(M,N);  T(M,N)  for 
V(M,N);  IT(M,  N)  for  IV(M,N)  and  JT(M,N)  for  JV(M,N). 

13.  Search  all  of  list  V for  its  minimum  value;  i.  e.  , set 

VMIN  = min  {V(I,  J);  V(I,J)  > oj 
all  I 
and  J 

Record  MS,  NS  as  the  values  of  I,  J yielding  VMIN. 

14.  Repeat  step  13  to  obtain  TMIN  from  list  T.  Record  MU,  NU 
as  the  values  of  1,J  yiek.1  TMIN. 

15.  If  the  best  route  selected  in  steps  13  and  14  is  one  that  contains 

an  attack  segment;  i.  c. , if  TMIN  < VMIN,  go  to  step  20;  otherwise, 
continue. 

16.  Record  the  threat  of  the  optimal  path  to  MS,  NS;  i.  e.  , 

set  U(MS,  NS)  = VMIN.  Set  V(MS,NS)  = 0 to  maintain  disjoint 
sets  and  set  I = MS  and  J = NS  as  the  next  point  to  be  analyzed 
by  the  procedure. 

17.  If  MS,  NS  is  an  entry  point  of  an  attack  segment;  i.  e.  , if 
MS  > IMAX,  go  to  step  19;  otherwise,  continue. 

18.  Set  ESCAPE  = . FALSE,  to  indicate  that  the  new  route  docs  not 
contain  an  attack  segment  and  go  to  step  4. 

19.  Set  ESCAPE  = . TRUE,  to  indicate  that  the  route  now  contains 
an  attack  segment.  Also  record  the  threat  posed  by  the  route  in 
the  set  reserved  for  routes  containing  attack  segments;  i.  e.  , 
set  S(MS,  NS)  - VMIN;  go  to  step  4. 

20.  Record  the  threat  of  the  optimal  path  to  MU,  NU;  i.  e.  , set 
S(MU,NU)  = TMIN.  Set  T(MU,  NU)  = 0 to  maintain  disjoint 
sets  and  set  I = MU  and  J = NU  as  the  next  point  to  be  analyzed 
by  the  procedure. 
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21. 


If  MU,  NU  is  on  one  of  the  terminal  boundaries  of  the  route; 
i.  e.  , if  MU  = NBOUND(K),  K = 1,. . . , NBD,  go  to  step  22; 
otherwise,  set  ESCAPE  = . TRUE,  and  go  to  step  4. 

22.  The  route  is  complete.  Trace  backward  through  the  route  using 
IT,  JT,  IV  and  JV  to  locate  all  the  points  comprising  the  route. 

The  position  coordinates  of  all  route  points  except  the  attack 
segment  entry  point  are  given  by  the  arrays  XLOCH,  YLOCH. 

The  attack  entry  point  position  is  found  in  the  arrays  XENTRY, 
YENTRY. 

The  procedure  outlined  above  uses  two  internal  procedures  that  are  very 
important  and  they  must  be  discussed.  The  procedure  used  to  determine  a 
route  segment  threat  DUMN  is  so  important  its  discussion  is  deferred  until  the 
next  chapter.  However,  the  procedure  for  mechanizing  the  search  for  neighbor 
points  is  discussed  in  the  following  paragraphs. 

The  procedure  used  to  search  for  neighbor  points  (M,N)  to  a given  route 
analysis  point  (I,  J)  is  designed  so  that  the  last  neighbor  point  it  determined  is 
input,  and  it  then  determines  the  next  neighbor  point.  The  procedure  is  pro- 
grammed as  a subroutine  to  be  called  by  the  route  selection  computer  pro- 
gram as  needed.  The  procedure  is  initialized  by  providing  (I,J)  as  input. 

When  all  neighbors  of  (I,  J)  have  been  determined,  then  (I,  J)  will  be  returned. 

This  fact  is  used  to  terminate  the  search  for  neighbors. 

Now,  if  (I,J)  is  a point  on  one  of  the  intervisibility  area  boundaries 

(I  — IMAX)  and  if  a route  that  does  not  include  an  attack  segment  is  being 

analyzed  (ESCAPE  = .FALSE.)  then  the  procedure  searches  all  points  on  all 

intervisibility  line  segments  and  all  attack  entry  points  for  the  neighbors  of 

(I,J).  The  search  is  conducted  by  first  incrementing  J on  line  I until  all  points 

on  line  1 have  been  analyzed.  Then  l Is  incremented  and  J is  reinitialized  at 

(?1 


r 

r 


one.  When  all  I have  been  examined,  I is  reinitialized  at  one  and  the  procedure 
is  continued  until  the  original  I,J  combination  is  again  achieved.  To  be  a 
neighbor  of  (I,J),  the  point  (M,N)  must  satisfy  one  of  the  following  sets  of  rela- 


tionships 


NFL(I)  = NFL(M) 
or  NFL(I)  = NFR(M)  l 
or  NFR(I)  = NFL(M) 
or  NFR(I)  = NFR(M) 


if  M SiMAX 


or 


NFL(I)  = NFRL(M,N) 
or  NFL  (I)  = NFRR(M,N) 
or  NRF(I)  = NFRL(M,N) 
or  NFR(I)  = NFRR(M,N) 


if  M > IMAX 


The  first  set  of  relations  is  valid  when  M is  not  an  attack  ray  and  the  second  set 
is  valid  when  M is  an  attack  ray.  The  quantities  used  were  described  on  page  51 
of  this  chapter  and  identify  the  intervisibility  areas  adjacent  to  a point. 

If  (I,J)  is  a point  as  above,  but  the  route  already  contains  an  attack  seg- 
ment; i.e. , if  ESCAPE  = .TRUE.  , then  the  search  for  neighbors  is  conducted 
in  the  same  order  described  above,  but  attack  entry  points  are  excluded  since 
only  one  attack  is  permitted  per  route.  Therefore,  no  neighbors  with  M > IMAX 
are  considered.  Consequently,  the  second  set  of  relations  above  is  not  required. 

If  (I , J)  is  an  attack  exit  point  then  the  procedure  is  similar  to  that  descri- 
bed above  for  the  case  when  ESCAPE  = .TRUE. , since  this  is  the  case  here. 
However,  the  order  of  search  is  somewhat  altered  since  it  is  unnecessary  to 
increment  J on  the  original  line  segment  I.  Only  one  point  per  attack  ray  is 
considered  by  definition.  Instead,  I and  J are  immediately  reinitialized  to  one 


to  begin  the  search  for  neighbors  among  the  intervisibility  boundary  lino  segments. 
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CHAPTER  3 


PHASE  I EXPERIMENTATION 


by  Don  C.  Hutcherson 


Introduction 

In  Chapter  2 a model  was  derived  to  predict  the  decision-making  behav- 
ior of  helicopter  pilots  in  a route  planning  task.  It  was  proposed  that  a pilot 
selects  a route  plan  in  such  a way  as  to  minimize  "exposure"  to  the  enemy,  and 
to  represent  this  mode  of  behavior,  the  concept  of  "perceived  threat"  was  intro- 
duced. This  concept  was  'ised  in  the  decision  rule  as  follows: 

From  a set  of  alternative  route  plans,  select  as  the  plan 
to  be  followed  that  which  possesses  the  least  perceived 
threat. 

The  procedure  employed  for  constructing  route  alternatives  produces 
routes  that  consist  of  a series  of  connected  straight-line  segments.  Thus,  the 
perceived  threat  of  a route  alternative  is  actually  some  function  of  the  per- 
ceived threats  possessed  by  each  of  the  segments  within  the  route.  Assuming 
utility  independence  and  incremental  increases  in  threat  always  increase  route 
threat  (Fishburn  and  Keeney,  1975),  a linear  function  was  proposed.  That  is, 
if  tj  is  the  perceived  threat  of  alternative  a^,  then 

f Ni 


ti  = < 2 t„ 

1 Sj 


where  ts  is  the  perceived  threat  of  segment  j,and  N|  is  the  number  of  segments 
comprising  alternative  aj. 
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The  most  demanding  task  of  the  research  program  came  after  the  deci- 
sion model  had  been  derived,  for  it  was  then  necessary  to  develop  procedures 
to  be  used  in  validating  the  model.  As  outlined  in  Chapter  1,  the  validation 
effort  actually  was  comprised  of  two  experiments.  The  first,  known  as  the 
Phase  I Experiment  was  designed  to  gather  data  for  the  model.  The  second, 
known  as  the  Phase  II  Experiment,  was  designed  to  test  the  predictive  qualities 
of  the  model  based  on  data  collected  during  Phase  I.  In  this  chapter,  attention 
is  directed  to  a comprehensive  description  of  the  Phase  I Experiment  and  re- 
sults obtained  at  Ft.  Rucker. 

One  of  the  first  and  most  critical  research  tasks  associated  with  Phase 
I was  development  of  methods  for  eliciting  values  of  ts  from  individual  deci- 

i 

sion  makers.  Without  this  capability  it  would  have  been  impossible  to  use  the 
route  planning  model  to  predict  routes  for  those  decision  makers  and,  hence, 
to  validate  the  model.  The  task  was  made  even  more  difficult  by  the  fact  that 
very  little  was  known  about  the  way  in  which  a decision  maker  actually  views 
the  relationship  between  t„  and  all  the  situation  variables  associated  with  route 

Sj 

segment  j.  Therefore,  not  only  was  a procedure  for  measurement  required, 
but  also  a model  (or  models)  of  perception  was  needed. 

The  second  major  task  associated  with  Phase  I was  development  of  an 
experimental  design  to  be  used  in  gathering  data.  The  design  had  to  yield  the 
required  information  efficiently,  for  the  experiment  was  to  be  conducted  with 
human  subjects , and  it  was  desired  that  subject  fatigue  not  become  a factor  in 
the  experiment. 


64 


The  next  Phase  I research  task  consisted  of  locating  a suitable  subject 


I 
I 

group  and  of  conducting  the  required  experiment.  Since  it  was  desired  that 
qualified  decision-makers  be  used  as  subjects,  the  search  was  limited  to  Army 
helicopter  pilots  with  combat  experience.  With  this  limitation  it  was  considered 
most  convenient  to  conduct  the  experiment  at  an  Army  installation  at  which  a 
large  population  of  prospective  subjects  might  be  found. 

The  final  Phase  I research  task  was  associated  with  analysis  of  data 
gathered  during  the  experiment.  During  this  task,  data  for  the  Phase  II  Exper- 
iment were  developed,  and  some  preliminary  conclusions  about  the  decision- 
making behavior  of  pilots  were  formulated. 

In  this  chapter,  all  the  topics  discussed  above  will  be  described  at 
length.  First,  a short  discussion  is  presented  in  which  pertinent  route  selec- 
tion situation  variables  are  identified.  This  discussion  will  also  show  how  a 
decision-maker  might  relate  perceived  threat  to  these  variables.  Then,  since 
it  is  unknown  how  a decision-maker  views  the  relationship  in  an  actual  route 
planning  task,  three  models  are  proposed  to  represent  a spectrum  of  anticipated 
behavior.  Next,  the  experimental  design  developed  to  gather  the  appropriate 
data  is  discussed,  and  is  followed  by  a description  of  measurement  procedures 
used  to  gather  the  required  data  from  decision-makers.  Finally,  selection  of 
the  subject  group,  conduct  of  the  experiment  and  analysis  of  the  results  are 


presented  as  separate  topics. 


Situation  Variables 

We  presume  that  the  threat  a pilot  feels  as  he  traverses  a given  route 
segment  may  first  be  a function  of  whether  or  not  he  is  exposed  to  an  enemy 
weapon.  The  perceived  threat  is  probably  much  lower  for  an  unexposed  route 
segment . 

Given  that  a segment  is  exposed,  the  perceived  threat  may  then  be  a 
function  of  factors  related  to  the  physical  characteristics  of  the  intervisible 
enemy  weapon  and  of  the  helicopter,  and  to  the  variables  describing  the  spatial 
orientation  of  the  helicopter  relative  to  the  enemy  weapon.  To  be  more  explicit, 
the  intensity  of  the  threat  might  be  described  as  a function  of  at  least  the  fol- 
lowing : 

1.  the  pilot's  assessment  of  the  volume  of  fire  of  which  the  weapon 
is  capable; 

2.  the  pilot's  assessment  of  the  lethality  of  accurately  aimed  fire 
from  the  weapon;  and 

3.  the  pilot's  assessment  of  the  accuracy  with  which  fire  may  be 
delivered. 

All  three  of  the  above  characteristics  are  properties  of  the  opposing 
weapon  system.  However,  the  latter  two  are,  to  an  extent,  controllable  by  the 
pilot.  For  example , both  lethality  and  accuracy  are  functions  of  the  range  to 
the  opposing  weapon  and  the  orientation  of  the  helicopter  relative  to  the  weapon. 
Thus,  the  pilot  can  affect  lethality  and  accuracy  by  his  choice  of  a route  in  rela- 
tion to  the  enemy's  position. 
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To  better  understand  the  above  discussion,  let  us  specify  the  geometry 


of  a threat  evaluation  situation  as  in  Figure  6.  The  segment  is  of  length  D and 
its  midpoint  is  separated  from  an  enemy  weapon  by  a distance  R.  Furthermore, 
the  helicopter  is  heading  at  an  angle  4>  relative  to  the  weapon.  If  the  helicopter 
is  proceeding  at  a speed  V,  then  the  exposure  time  while  traversing  the  segment 
is  T = D/V. 

The  perceived  accuracy  of  enemy  fire  may  be  a complex  function  of  V 
and  of  all  the  geometric  variables  shown  in  Figure  6.  In  general,  increasing 
range  might  tend  to  reduce  perceived  accuracy,  increased  exposure  time  might 
tend  to  increase  perceived  accuracy,  and  changes  in  4>  might  affect  perceived 
accuracy  by  changing  the  profile  presented  to  the  weapon.  Perceived  weapon 
lethality  might  also  be  partially  a function  of  range  and  of  4> . Therefore , from 
all  previous  discussions,  it  is  presumed  that  the  following  quantitative  and 
qualitative  situation  variables  are  important  enough  to  be  considered  in  the 
analysis: 


Quantitative: 

R - range  to  enemy  weapon 
T - exposure  time 
<)>  - relative  heading  angle 
V - helicopter  speed 

Qualitative : 

C„  - pilot  self  confidence 

O 

CfI  - pilot  confidence  in  his  vehicle 
W - enemy  weapon  type. 


i 
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Figure  6. — Threat  Evaluation  Situation 
Now,  the  effects  of  factors  Cg,  Cjj  , and  V are  lumped  into  one  effect 
which  is  called  the  subject  effect.  The  factor  that  produces  this  effect  is 
defined  as  S and  serves  as  a representation  of  differences  among  pilots.  In 
the  research  program  it  has  not  been  deemed  necessary  to  study  the  effects  of 
Cjj  and  Cg  individually.  Instead,  the  overall  subject  effect  S is  considered 
to  be  the  important  variable.  However,  it  has  been  our  experience  that  heli- 
copter pilots  are,  in  general,  accustomed  to  operating  a particular  type  of 
vehicle  in  a given  mission  situation.  Therefore,  during  experimentation,  the 
CR  effect  becomes  confounded  with  the  Cg  effect  anyway.  Finally,  the  speed 
V with  which  a route  segment  will  be  traversed  is  partially  a personal  choice. 
However,  it  also  is  a property  of  the  vehicle  type  which  the  pilot  implicitly 
assumes  himself  to  be  flying  in  a given  mission  context.  Therefore,  the  effect 
of  V can  be  (and  is)  lumped  in  the  total  effect  of  S.  Hence,  the  final  set  of  situ- 


Perceived  Threat  Models 


In  the  previous  section,  commentary  was  offered  on  how  a decision- 
maker might  view  the  relationship  between  perceived  threat  and  the  situation 
variables  discussed.  However,  very  little  is  known  about  the  decision-maker's 
actual  mode  of  thought.  The  arguments  used  were  intuitive  and  are  certainly 
subject  to  error.  Therefore,  in  the  absence  of  more  information,  three  models 
of  the  relationship  are  proposed.  While  the  exact  form  of  each  model  is  to  an 
extent  arbitrary,  the  models  are  intended  to  cover  a wide  spectrum  of  possible 
behavior.  The  experimentation  plan  discussed  in  Chapter  4 has  been  designed 
in  such  a way  that  the  model  or  models  that  are  most  representative  of  individual 
pilot  decision  behavior  in  an  actual  route  planning  task  can  be  isolated. 

The  first  model  is  known  as  Model  One.  It  is  intended  to  be  represen- 
tative of  a class  of  perception  models  which  are  almost  insensitive  to  differences 
among  individual  pilots.  Simply  stated,  the  threat  perceived  by  a pilot  on  a 
route  segment  is  predicted  by  the  relation: 

✓ 

N • T if  tne  segment  is  exposed  to  N enemy  weapons 

ls  = ‘ 

.001  • T if  the  segment  is  unexposed 

where  T is  the  travel  time  for  a given  route  segment.  Note  that  the  model  is 
sensitive  to  individual  variance  only  because  different  pilots  select  different 
speeds  with  which  to  traverse  a path.  Note  also  that  for  simplicity  the  model 
assumes  additive  disutilities  with  respect  to  exposure  to  multiple  enemy 
weapons. 
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In  several  ways.  Model  One  represents  conservative  decision  making. 


First,  exposure  to  even  one  enemy  weapon  for  a given  period  of  time  is  pre- 
dicted to  be  perceived  as  much  more  threatening  than  non-exposure  for  the  same 
period.  The  factor  of  one-thousand  has  been  assumed  arbitrarily  in  Model  One 
to  account  for  this  type  of  conservatism.  Second,  no  matter  what  the  range  to 
a weapon  is,  the  weapon  is  still  considered  as  threatening  as  any  other  weapon. 
Finally,  all  weapons  regardless  of  their  characteristics  are  considered  equally 
threatening. 

In  other  ways,  the  decision  behavior  is  not  so  conservative.  Note  that 
among  equally  exposed  route  segments  (including  non-exposed  route  segments) , 
Model  one  predicts  that  a pilot  will  select  the  shortest  route . This  prediction 
is  made  without  regard  to  the  range  or  the  orientation  of  the  segment  with  respect 

to  any  of  the  weapons  to  which  the  route  might  be  exposed.  Thus,  a short  exposed 

* 

segment  passing  very  close  to  any  enemy  weapon  woUld  be  chosen  over  a longer 
exposed  segment  that  is  more  distant  from  the  weapon.  Such  behavior  fails  to 
account  for  the  fact  that  the  weapon  might  be  considerably  more  lethal  at  close 
range.  Also,  the  model  does  not  account  for  the  fact  that  some  weapons  are 
more  accurate  if  fired  from  the  helicopter’s  side  aspect  while  others  are  more 
accurate  from  the  rear.  Nevertheless , since  protected  route  segments  are 
favored  so  strongly  in  comparison  to  exposed  segments,  the  model  does  repre- 
sent conservative  behavior.  In  fact,  the  model  may  very  well  be  representative 
of  many  types  of  decision  makers. 
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Experimentation  required  to  gather  input  data  for  the  model  consists 
merely  of  determining  what  speeds  a particular  helicopter  pilot  will  employ 
during  different  phases  of  his  mission.  Since  the  model  is  so  insensitive  to 
individual  behavior,  one  might  view  the  model  as  one  to  be  used  in  the  absence 
o'  any  information  regarding  individual  pilot  decision  behavior. 

The  next  model  down  on  the  scale  of  conservatism  and  up  on  the  scale 
of  sensitivity  is  a model  that  is  known  as  Model  Three.1  This  model  states 
that  the  threat  percieved  by  a pilot  on  a route  segment  can  be  predicted  by  the 
relation 


t 


s ~ 


N 

T • 2 6j  Cj  if  the  segment  is  exposed  to  at  least  one 
1-1  enemy  weapon 

.001  * T if  the  segment  is  unexposed 


where 

1 if  the  midpoint  of  the  route  segment  is  within  the  effective 
firing  range  of  enemy  weapon  i and  is  exposed  to  weapon  i 

0 if  otherwise. 

Cj  = the  relative  threat  of  weapon  i with  respect  to  the  weapon  type 
being  attacked 

N = total  number  of  enemy  weapons  considered  in  the  tactical  scenario. 


Model  Two  was  dropped  during  experimentation  when  it  was  found  that 
it  produced  results  that  were  almost  always  identical  to  those  produced  by 
Model  One. 
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Again,  the  exact  form  of  the  relation  is  somewhat  arbitrary.  It  is  intended 
merely  to  reflect  a relaxation  in  conservatism  and  a higher  degree  of  sensitivity 
to  differences  between  individual  pilots.  For  simplicity,  additive  disutilities 
is  again  assumed  with  respect  to  exposure  to  multiple  enemy  weapons.  Also, 
the  relative  threats  of  enemy  weapons  (C^)  and  the  maximum  effective  range  of 
enemy  weapons  are  quantities  to  be  determined  from  individual  pilots.  They  are 
subjective  estimates  as  opposed  to  quantitative  weapon  performance  estimates. 
Finally,  the  perceived  threat  of  weapon  i is  measured  relative  to  the  perceived 
threat  of  the  target  weapon  type  which  is  scaled  to  one.  Thus,  this  model  would 
yield  the  same  results  as  Model  One  if  the  maximum  effective  range  of  all  enemy 
weapons  were  judged  to  be  infinite  and  if  Cj  = Cg  = . . . = = 1. 

Model  Three  is  less  conservative  than  Model  One  in  that  enemy  weapons 
are  considered  threatening  only  if  the  route  segment  is  perceived  to  be  within 
range  of  the  weapons.  Furthermore,  the  existence  of  perceived  differences 
between  weapons  is  acknowledged — not  all  weapons  are  considered  equally 
threatening.  The  model  still  does  not  recognize  the  effects  that  orientation  of 
a route  segment  with  respect  to  a given  enemy  weapon  might  have  on  the  per- 
ceived threat;  nor  does  the  model  recognize  variations  of  perceived  threat  as  a 
function  of  range  within  the  maximum  effective  range  perceived  for  a given 
weapon.  The  ramifications  of  these  model  properties  were  outlined  earlier 
in  the  discussion  of  Model  One. 
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The  model  is  more  sensitive  to  variations  in  individual  behavior 
because  the  relative  threats  of  enemy  weapons  (C^)  and  the  maximum  effective 
range  of  weapons  are  subjective  estimates  as  opposed  to  quantitative  estimates. 
Thus,  to  use  this  model,  procedures  must  be  developed  to  elicit  range  and  threat 
information  from  actual  decision  makers.  When  Model  Three  is  used  to  predict 
t,  the  routes  selected  by  the  model  derived  in  Chapter  2 become  more  a property 
of  an  individual  pilot. 

The  final  model,  or  Acid  Test  Model,  derives  its  name  from  the  fact 
that  it  is  associated  with  the  most  detailed  and  complex  representation  of  pilot 
behavior  considered.  It  is  the  most  sensitive  of  the  three  models  to  individual 
decision-maker  behavior.  As  it  turns  out,  it  also  represents  the  least  conser- 
vative behavior  of  the  three  models.  Routes  selected  using  this  model  to  pre- 
dict route  segment  threat  can  be  associated  with  very  aggressive  pilot  perfor- 
mance. 


The  Acid  Test  Model  represents  behavior  corresponding  to  that  pro- 


posed in  the  preceding  section.  Thus,  it  predicts  route  segment  threat  per- 


ception with  the  following  relation 


t 


s 


N 

2 • Di(R,  4>,  T)  if  the  segment  is  exposed  to 

at  least  one  enemy  weapon 

F(T)  if  the  segment  is  unexposed 


where 


6i  - 


1 if  weapon  i is  intervisible  with  the  midpoint  of  the 
route  segment 
0 if  otherwise 


! 
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R = range  to  the  enemy  weapon  from  the  segment  midpoint 

4>  = included  angle  between  the  segment  heading  vector  and  a 
vector  to  the  target  from  the  segment  midpoint 

T = travel  time  on  the  segment 

N = total  number  of  weapons  considered  in  the  tactical  scenario 

Dj(R,  <j>,  T)  = empirically  determined  function  of  R,  4>,  and  T, 
relating  threat  of  the  segment  to  weapon  i 

F(T)  = empirically  determined  function  relating  unexposed  travel 
time  to  perceived  segment  threat. 

The  reader  should  note  that  the  functions  F(T)  and  Dj(R,  4>,  T)  are  actually 
decision-maker  opinions.  Thus,  the  model  is  subjectively  based  in  its  entirety. 
As  related  in  a following  section,  the  procedures  developed  to  elicit  these  decision- 
maker opinions  give  rise  to  a major  experimentation  effort.  Furthermore, 
analysis  of  data  reveals  that  Dj(R,  <j>,  T)  is  very  much  the  property  of  an 
individual  decision-maker.  However,  it  is  determined  that  the  magnitude  of  the 
range  of  F(T)  in  relation  to  the  range  of  D^(R,  4>,  T)  allows  one  to  use  prac- 
tically any  small- valued  positive  function  for  F(T). 

Additivity  Implications 

The  reader  will  recall  from  Chapter  2 that  an  assumption  basic  to  the 
entire  route  selection  model  is  that  additive  disutilities  exist  for  segments 
of  a route  (with  respect  to  perceived  threat).  To  test  this  assumption 
one  would  have  to  know  how  a decision-maker  actually  perceives  threat  as 
indicated  by  his  performance  in  a route  planning  task.  However,  the  three 
threat  perception  models  that  have  been  proposed  simply  describe  possible 
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modes  of  behavior.  Thus,  the  assumption  cannot  be  tested  in  the  Phase  I 
Experiment.  Nevertheless,  an  interesting  question  still  exists.  If  a decision- 
maker actually  did  perceive  threat  as  described  by  one  of  the  three  models, 
what  would  be  the  implications  insofar  as  the  additivity  assumption  is  concerned? 

With  certain  restrictions,  if  a decision-maker  perceives  threat  accord- 
ing to  Model  One  or  according  to  Model  Three,  then  the  assumption  of  additive 
segment  disutilities  is  satisfied.  This  is  so  because  perceived  threat  is  a simple 
linear  function  of  T only.  However,  if  a decision-maker  actually  perceived 
threat  according  to  the  Acid  Test  Model,  then  it  is  highly  unlikely  that  the 
assumption  would  be  valid  since  very  special  forms  of  the  function  T) 

would  be  required.  For  example,  if  D^R,  <j>,  T)  were  invariant  in  R and  4>  but 
non-linear  in  T,  the  assumption  would  be  violated.  Of  course,  if  the  function 
were  also  linear  in  T,  then  Model  One  would  result  and  the  assumption  would 
be  satisfied. 

Thus,  one  of  the  purposes  of  the  Phase  I Experiment  was  to  determine 
whether  or  not  additive  segment  disutilities  could  be  assumed  for  a route, 
given  that  a decision-maker  actually  does  perceive  threat  according  to 
the  Acid  Test  Model.  Such  independence  would  be  expected  if  the  decision-maker 
actually  considers  his  conditional  probability  of  survival  on  any  segment  to  be 
independent  of  that  on  the  other  segments,  and  if  the  relationship  between  this 
survival  probability  and  perceived  threat  is  in  fact  that  proposed  in  Chapter  2. 


^Discussed  in  Chupters  4 and  7. 
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As  will  be  seen  in  the  analysis  section  of  this  chapter,  no  special  data  had  to  be 
collected  for  the  test  of  independence. 

Experimental  Design 

The  design  developed  for  the  Phase  I Experiment  had  to  be  extensive 
enough  to  gather  data  required  by  each  of  the  perceived  threat  models  described 
earlier  in  this  chapter.  Moreover,  data  were  desired  to  check  some  of  the 
assumptions  of  the  route  selection  model  in  general  and  of  the  perceived  threat 
models  in  particular.  Thus,  the  experiment  was  designed  to  yield  the  following 
data  for  each  decision-maker: 

1.  the  type  of  helicopter  normally  flown  during 
an  attack; 

2.  the  speed  normally  employed  while  conducting 
knap-of-the-earth  flight  as  well  as  a range  of 
variation  in  this  speed; 

3.  the  speed  normally  employed  while  conducting  a 
running- fire  attack  as  well  as  a description  of  how 
speed  might  vary  during  an  attack; 

4.  the  overall  relative  threat  perceived  for  each  of 
several  different  types  of  enemy  weapons; 

5.  the  maximum  effective  range  perceived  for  each 
of  several  different  types  of  weapons; 

6.  an  estimate  of  the  function  Dj(R,4>  , T)  for  each 
of  several  different  types  of  weapons;  and 

7.  an  estimate  of  the  function  F(T). 

As  can  be  seen,  the  amount  of  information  to  be  gathered  from  each  subject  was 
quite  large.  Therefore,  the  design  for  the  Phase  I Experiment  had  to  be  efficient 
to  avoid  subject  fatigue. 
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As  discussed  in  the  next  section,  direct  interrogation  methods  were 
employed  to  obtain  all  of  the  data  outlined  above.  Thus,  the  only  data  for  which 
particularly  interesting  experimental  designs  were  required  are  the  data  used 
in  estimating  D^R,  <t>,  Tj  and  F(T).  A discussion  of  the  design  for  D^R,  <t>,  T) 
will  be  presented  first. 

Dj(R,  4»,  T).  The  function  Dj(R,4>,  T)  has  been  assumed  explicitly  to  be  dependent 
upon  the  route  segment  exposure  time  (T),  the  heading  of  the  helicopter  rela- 
tive to  the  enemy  weapon  (cf>),  the  distance  from  the  route  segment  mid-point 
to  the  enemy  weapon  (R),  and  the  type  of  enemy  weapon  being  considered  (i). 

In  addition,  it  has  been  stated  that  each  pilot  or  subject  might  be  unique  in 
his  assessment  of  enemy  threat.  Thus,  it  is  assumed  that  D^(R,  (j>,  T)  is  also 
a function  of  the  subject  to  whom  a given  D.(R,  <j>,  T)  belongs.  Finally,  it  is 
assumed  that  an  experiment  conducted  to  determine  D.  (R,  <j>,  T)  for  a given 
subject  would  yield  data  from  which  only  an  estimate  of  the  function  could 
be  obtained  since  each  observation  might  possibly  be  in  error.  Specifically,  it 
is  assumed  that  the  jth  response  obtained  from  a given  subject  and  correspond- 
ing to  a particular  set  of  values  for  i,  R,  4>»  and  T can  be  expressed  as 

O.  = n (R,  <j>,  T)  + 6. 

where  Oj  is  the  reponse  obtained  and  c j is  the  error  in  the  response.  Moreover, 

c j is  assumed  normally  and  independently  distributed  with  zero  mean  and  variance 

2 
or  . 
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Now,  aside  from  the  above  assumptions,  very  little  was  initially  known 
about  the  function  D^R,  cj>,  T)  that  could  be  used  to  guide  the  development  of  an 
experimental  design.  It  was  suspected  that  the  function  might  have  a very  com- 
plex shape,  and  that  it  might  vary  considerably  from  one  subject  to  another. 
Furthermore,  it  was  known  that  a fairly  complete  description  of  the  function 
would  be  required  for  each  subject  since  the  description  would  be  used  as  input 
data  for  the  route  planning  model.  Finally,  it  was  desired  that  the  experiment 
should  yield  a statistically  accurate  and  precise  estimate  of  the  function. 

From  these  considerations,  it  was  decided  to  conduct  a factorial  experi- 
ment that  would  comprehend  a wide  variation  in  each  of  the  factors  i,  R,  <j>  and 
T for  each  subject  participating  in  the  experiment.  However,  consideration  of  the 
relatively  large  set  of  independent  variables  necessitated  a very  cautious  selec- 
tion of  the  number  of  levels  of  each  factor.  Moreover,  even  with  comparatively 
straightforward  techniques  for  eliciting  responses  at  each  combination  of  factor 
levels,  subject  fatigue  was  contemplated  as  becoming  a possible  problem. 
Therefore,  the  design  allowed  for  only  one  replication  of  the  factorial  for  each 
subject.  Thus,  the  design  was  a (w)x(t)x(r)x(p)  factorial  with  one  replication 
per  subject  where 

w = number  of  levels  of  enemy  weapon  factor, 
t = number  of  levels  of  the  route  segment  exposure  time  factor, 
r number  of  levels  of  the  separation  range  factor,  and 
p number  of  levels  of  the  relative  heading  laclor. 


7H 


AD-A045  094 


UNCLASSIFIED 


OHIO  STATE  UNI V COLUMBUS  SYSTEMS  RESEARCH  GROUP  F/G  1 

THE  HELICOPTER  ROUTE  SELECTION  MODEL  (DYNFLITE) . (U) 

AUG  76  G M CLARK*  D C HUTCHERSON*  0 C BITTERS  DAAG25-70-C-031I 
RF-2978— FR-76-4 ( 0 ) NL 


Now,  for  purposes  of  analysis,  there  was  actually  a fifth  factor  in  the 
experiment  to  account  for  the  assumption  stated  previously  that  each  subject  might 
be  unique  in  his  assessment  of  (R,  <j>,  T).  This  factor  is  called  the  subject  fac- 
tor and  was  measured  at  s levels,  where  s is  the  number  of  subjects  that  par- 
ticipated in  the  experiment.  Thus,  the  overall  design  was  actually  a 
(s)x(w)x(t)x(r)x(p)  factorial  with  one  replication.  Another  approach  would  have 
been  to  assume  that  there  was  no  subject  effect.  Had  this  assumption  been  adopted, 
then  the  design  could  have  been  viewed  as  a (w)x(t)x(r)x(p)  factorial  with  s replica- 
tions, and  the  precision  of  estimates  obtained  from  the  experiment  would  have 
been  controllable  through  the  parameter  s.  However,  the  assumption  of  no 
subject  effect  was  viewed  as  being  unduly  restrictive.  It  was  considered 
desirable  that  tests  for  the  existence  of  a subject  effect  be  conducted.  Then, 
if  such  tests  proved  negative,  the  alternative  model  still  could  be  adopted. 

After  the  basic  design  was  formulated,  attention  was  directed  to  an 
analysis  of  each  factor  in  the  experiment  to  determine  the  number  of  levels 
to  be  considered,  whether  the  factor  should  be  treated  as  quantitative  or 
qualitative,  and  whether  the  effect  associated  with  each  factor  should  be  treated 
as  fixed  or  random.  The  analysis  began  with  the  subject  factor  s. 

The  subject  factor  was  obviously  a qualitative  factor,  but  at  the  time 
the  design  was  being  formulated,  the  number  of  subjects  that  would  be  available 
for  the  experiment  was  unknown.  However,  as  it  turned  out,  eight  subjects 


effect. 
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Selection  of  w was  guided  by  a desire  to  obtain  information  across  a 
representative  cross  section  of  enemy  antiaircraft  weapons  that  might  be 
encountered  by  helicopters  in  a forward  area  during  a mid- intensity  war.  A 
review  of  various  classified  military  intelligence  documents  revealed  that  this 
objective  could  be  achieved  by  considering  three  types  of  weapons.  Thus,  the 
qualitative  factor  W was  measured  at  three  levels  and  was  considered  to  pro- 
duce a fixed  effect. 

The  weapons  selected  were:  (1)  the  14.  5 mm  light  antiaircraft  weapon 
system  employing  a manual  sighting  mechanism;  (2)  the  23  mm  light  antiair- 
craft weapon  system  employing  a radar  directed  firing  system;  and  (3)  the 
REDEYE  heat  seeking  guided  missile  system  launched  by  a dismounted  two  man 
crew.  All  weapons  are  highly  mobile  and  are  employed  in  forward  areas.  The 
14.  5 mm  weapon  is  considered  to  be  among  the  lightest  antiaircraft  weapons 
that  might  be  encountered  by  low  level  aircraft  such  as  helicopters  while  the 
other  two  weapons  might  be  among  the  heaviest  to  be  encountered. 

Selection  of  the  levels  of  the  remaining  factors  T,  R and  4>  was  guided 
by  a desire  to  determine  an  "adequate"  definition  of  the  response  surface  for 
each  weapon  and  subject.  This  meant  that  the  levels  selected  should  cover 
the  expected  range  of  variation  of  each  factor,  and  that  enough  levels  of  each 
factor  should  be  selected  to  provide  an  indication  of  the  shape  of  the  surface 
as  a function  of  each  factor.  However,  the  number  of  levels  of  each  factor  that 
could  be  used  was  necessarily  limited  by  the  endurance  of  the  subjects  and  the 
time  available  for  the  experiment.  Thus  it  was  decided  to  use  a minimum  of 
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three  levels  and  a maximum  of  five  levels  as  explained  in  the  following  para- 
graphs. With  a minimum  of  three  levels  available  for  each  factor  it  was 
reasoned  that  an  estimate  of  up  to  a quadratic  effect  would  always  be  available 
for  any  one  of  the  three  factors,  and  this  was  considered  to  provide  an 
"adequate"  definition  of  the  response  surface. 

Selection  of  the  levels  of  T was  guided  by  a desire  to  cover  the  range  of 
exposure  times  that  might  be  encountered  while  traversing  a single  route  seg- 
ment of  the  type  discussed  in  Chapter  2.  Therefore,  an  analysis  of  represen- 
tative route  selection  maps  of  the  type  shown  in  Figure  5,  Chapter  2 was 
conducted.  It  was  found  that  45  seconds  would  be  a suitable  upper  limit  on  T. 

It  was  also  decided,  arbitrarily,  that  the  minimum  number  of  levels  (three) 
discussed  above  would  be  sufficient.  Thus,  the  quant  factor  T was  con- 

sidered to  be  a fixed  effect  at  three  levels.  The  levels  c^sen  were  5,  15,  and 
45  seconds.  A fourth  level  of  T at  zero  seconds  was  implied.  However,  at  this 
level,  subjects  would  presumably  feel  no  threat,  hence  the  response  surface 
would  collapse  to  zero.  Therefore,  zero  was  not  employed  as  a level  of  T. 

Selection  of  levels  of  R was  guided  by  a desire  to  cover  the  range  of 
separation  distances  for  which  a decision  maker  might  feel  threatened.  One 
value  most  certainly  is  the  value  zero.  Indeed,  a decision  maker  might  feel 
most  threatened  in  passing  directly  over  a weapon.  The  other  value  is  the 
distance  at  which  the  perceived  threat  first  falls  to  zero.  However,  this  dis- 
tance was  unknown  at  the  time  that  the  experiment  was  being  designed.  Further- 
more, it  was  anticipated  that  this  zero  threat  range  would  be  a function  of  the 
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subject,  the  weapon  type  and  the  exposure  time,  and  might  even  be  a function 
of  relative  heading.  Thus,  the  zero  threat  range  could  not  be  used  directly 
as  a factor  level  in  the  perceived  threat  experiment. 

To  solve  the  above  dilemma,  it  was  decided  simply  to  select  three 
evenly  spaced  levels  of  R that  would  probably  fall  within  the  zero  threat  range 
but  still  cover  most  of  the  range  of  separation  distances  for  which  a decision 
maker  might  feel  threatened.  It  was  decided  also  that  a separate  experiment 
should  be  designed  to  obtain  zero  threat  range  data.  By  adopting  this  approach, 
it  was  reasoned  that  the  minimum  acceptable  degree  of  surface  definition  dis- 
cussed above  would  be  achieved.  Moreover,  the  zero  threat  range  data  would 
be  available  to  provide  a more  complete  description  of  the  response  surface  as 
a function  of  R.  Hence,  in  the  perceived  threat  experiment,  R was  considered 
a quantitative  factor  measured  at  three  levels  yielding  a fixed  effect.  The  three 
levels  were  specified  as  follows: 

1.  a separation  of  zero  meters; 

2.  a separation  that  is  approximately  40%  of  the  estimated 
effective  range  of  the  weapon  as  published  in  weapon 
documentation;  and 

3.  a separation  that  is  approximately  80%  of  the  published 
effective  range. 

The  experimental  design  used  to  obtain  zero  threat  range  data  will  be  discussed 
in  a paragraph  to  follow. 

The  final  factory  was  considered  to  produce  a fixed  effect  measured  at 
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This  range  of  variation  was  selected  to  cover  all  possible  variations  in  relative 
heading.  However,  the  number  of  levels  was  selected  somewhat  arbitrarily. 

It  was  felt  that  more  than  three  values  would  be  required  to  adequately  cover  the 
range  of  variation  of  4>  , and  with  five  levels,  the  actual  angular  values  used 
become  intuitively  satisfying. 

Now,  to  have  obtained  a complete  replication  of  the  perceived  threat 
factorial,  135  observations  of  perceived  threat  would  have  had  to  be  obtained 
from  each  subject  (3x3x3x5).  However,  in  the  interest  of  a shorter,  more 
manageable  experiment,  it  was  decided  to  obtain  only  a fraction  of  the  complete 
replication.  Instead  of  eliciting  five  values  of  perceived  threat  from  each 
subject  at  R = 0 for  each  weapon  and  exposure  time,  only  one  value  was  obtained. 
This  procedure  then  permitted  an  experiment  consisting  of  only  99  observations 
from  each  subject  [(5  levels  of<f>)x  (2  levels  of  R)x  (3  levels  of  T)x  (3  levels 
of  W)plus(l  level  of  R)x  (3  levels  of  T)x  (3  levels  of  W)].  Thus,  an  incomplete 
factorial  design  was  actually  employed  in  the  experiment;  and,  as  discussed  in 
the  analysis  section  of  this  chapter,  some  problems  had  to  be  overcome  before 
satisfactory  analyses  could  be  performed  upon  the  data. 

The  design  to  obtain  zero  threat  range  data  was  again  a simple  factorial 
to  be  replicated  once  for  each  subject.  It  was  anticipated  that  there  would  be 
a subject  effect,  a weapon  effect,  an  exposure  time  effect  and  a relative  head- 
ing effect.  Thus,  the  design  was  a (s)x(w)x(t)x(p)  factorial  with  one  replication. 
For  simplicity,  the  levels  of  the  various  factors  used  in  the  perceived  threat 
experiment  were  also  used  in  the  zero  throat  range  experiment.  Hence,  the 
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factorial  involved  45  measurements  of  zero  threat  range  [(5  levels  of  4>)x  (3 
levels  of  t)x  (3  levels  of  W)]  from  each  subject. 


F(T).  To  obtain  data  to  estimate  F(T),  indifference  techniques  are  used  as 
explained  in  the  next  section.  Essentially,  the  procedure  consists  of  deter- 
mining the  amount  of  unexposed  travel  time  T a helicopter  pilot  would  be  willing 
to  endure  in  order  to  avoid  a specified  exposed  route  segment  situation.  Then, 
the  perceived  threat  of  the  exposed  route  segment  situation  becomes  a value  of 

F(T)  for  the  stated  unexposed  travel  time. 

It  was  suspected  beforehand  that  the  range  of  F(T)  would  be  rather  small 
in  comparison  to  the  range  of  D^R,*  , T).  Thus,  it  would  not  be  that  important 
to  determine  the  shape  of  F(T)  with  extreme  precision.  Therefore,  rather 
arbitrarily,  it  was  decided  to  determine  nine  data  points  from  each  subject 
with  which  to  estimate  F(T).  These  nine  points  came  from  a 3 x 3 fixed  effects 
factorial  involving  the  factors  W and  T,  with  the  same  levels  of  these  factors  being 
used  as  were  used  in  the  design  for  D.(R,4>  , T).  That  is,  W was  a qualitative 
factor  at  three  levels  (14.  5 mm  weapon,  23  mm  weapon,  and  REDEYE  weapon) 
and  T was  a quantitative  factor  at  three  levels  (5,  15,  and  45  seconds).  The 
factor  R was  fixed  at  its  second  level  (40%  maximum  effective  range)  and  4> 
was  fixed  at  its  third  level,  90°. 

Additive  Disutilities.  As  the  reader  may  recall,  each  of  the  three  perception 
models  assumes  additive  disutilities  with  respect  to  exposure  to  multiple 
enemy  weapons.  However,  no  provision  was  made  in  the  Phase  I 
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Experiment  to  test  this  assumption.  While  procedures  to  test  the  assumption 
are  relatively  simple  to  devise,  their  inclusion  would  have  involved  the  addition 
of  another  fairly  taxing  task  in  an  already  long  and  tiring  experimental  session. 
Moreover,  in  an  earlier  preliminary  experiment  involving  a different  subject 
group,  the  procedures  had  been  included.  While  no  statistical  analyses  were 
performed  on  the  resulting  data,  it  appeared  that  additive  disutilities  could 
indeed  be  assumed.  Thus,  in  view  of  the  preliminary  results  and  in  consider- 
ation of  the  extra  effort  that  would  have  been  involved,  procedures  to  gather 
data  for  testing  the  assumption  of  additive  disutilities  were  not  included. 

Method 

As  stated  previously,  methods  of  direct  interrogation  were  used  through- 
out the  Phase  I Experiment  to  obtain  data  from  subjects.  Again,  the  only  par- 
ticularly interesting  methods  were  those  associated  with  obtaining  data  for 
estimating  Dj(R,4>  , T)  and  F(T).  The  other  data  were  obtained  in  a rather 
straightforward  manner. 

Methods  to  be  used  to  measure  the  99  values  of  perceived  threat  and  the 
45  values  of  maximum  threatening  range  had  to  be  relatively  simple.  Even  with 
only  144  measurements,  there  was  the  possibility  of  subject  fatigue.  Moreover, 
the  subjects  had  to  be  able  to  understand  the  meaning  of  the  responses  they  were 
being  asked  to  make. 

**The  preliminary  experiment  was  conducted  among  personnel  of  the 
Ohio  Air  National  Guard.  See  Appendix  G for  a brief  deseription  of  the  exper- 
iment. 
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The  method  used  to  elicit  perceived  threat  values  from  a subject  for  each  fac- 
tor level  combination  was  a direct  interrogation  method  suggested  by  Dr.  James 
A.  Wise  of  The  Ohio  State  University  Department  of  Psychology.  The  method 
had  been  used  successfully  by  a number  of  experimental  psychologists  to  obtain 
numerical  data  in  a variety  of  subjective  evaluation  situations.  Furthermore, 
some  of  these  studies  had  yielded  data  that  could  be  measured  on  a ratio  scale, 
while  others  had  employed  an  interval  scale.  Notable  studies  had  involved  the 
measurement  of  subjective  probability  (Wise,  1970;  Mockovak,  1972);  however, 
a wide  variety  of  other  types  of  measurement  had  also  been  studied,  an  out- 
standing example  of  which  is  reported  by  Peterson  and  Beach  (1967).  These 
investigators  found  man  capable  of  estimating  fairly  well  all  manner  of  statis- 
tical quantities  including  proportions,  means,  variances,  and  correlations , 
both  of  samples  and  of  populations. 

The  problem  of  determining  perceived  threat  values  is  very  similar 
to  the  problems  cited  above.  That  is,  perceived  threat,  as  defined  in  this 
research,  is  a subjective  quantity  measurable  on  an  interval  scale.  This 
similarity  suggests  that  one  may  use  the  direct  measurement  technique,  and 
this  is  the  approach  that  was  followed.  However,  no  experimental  procedure 
was  included  to  test  the  psychophysical  aspects  of  the  approach.  All  that  can 
be  said  is  that  the  results  obtained  from  the  experiment  appear  reasonable . 

Now,  in  the  actual  experiment  subjects  were  asked  to  constrain  their 
subjective  threat  evaluations  to  the  interval  (0,  1000).  The  interval  used  is 
really  of  no  importance,  but  some  standard  was  needed.  It  was  assumed  that 
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subjects  would  find  zero  an  acceptable  value  to  be  associated  with  no  perceived 
threat.  An  interval  of  1000  was  considered  adequate  to  permit  a wide  range  in 
subject  response  if  the  subject  so  desired.  It  also  contains  enough  integer 
values  to  hopefully  prohibit  a subject  from  selecting  his  favorite  number  for 
an  answer  too  often.  This  last  property  of  the  interval  is  the  main  reason  that 
more  common  intervals  such  as  (0,  10)  or  (0,  100)  were  not  used  in  the  exper- 
iment. 

Briefly,  the  direct  interrogation  method  to  determine  perceived  threat 
consisted  of  a single  session  broken  into  several  segments.  The  segments  that 
were  involved  may  be  outlined  as  follows:  (1)  a briefing  of  the  subjects  to 
dc  them  (in  terms  familiar  to  military  aviators)  the  concept  of  per- 

.eat;  (2)  a briefing  to  acquaint  the  subjects  with  the  enemy  weapons 
to  which  they  would  be  exposed;  (3)  a review  of  the  range  of  variation  of  all  the 
geometric  situation  variables  to  which  they  would  be  exposed;  (4)  a segment 
in  which  each  subject  was  asked  to  determine  the  situation  or  situations  which 
he  felt  to  be  the  most  threatening  of  all  situations  to  be  examined;  (5)  instruc- 
tions to  assign  a maximum  perceived  threat  value  (1000)  to  any  situation  per- 
ceived to  be  equivalent  to  the  most  threatening  situation  determined  earlier; 

(6)  instructions  to  assign  zero  perceived  threat  to  any  situation  to  which  the 
subject  felt  no  threat;  and  (7)  a segment  in  which  perceived  threat  values  were 
elicited  in  response  to  each  of  the  factor  level  combinations  occurring  in  the 
design. 
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The  last  segment  employed  99  slides,  each  depicting  one  threat  situa- 


tion. As  the  slide  was  presented,  each  subject  was  asked  to  record  his  response 
on  a computer  data  card  which  was  prepunched  with  the  subject  number  and  the 
slide  number.  As  each  card  was  completed  it  was  placed  in  a container  so  that 
past  answers  could  not  be  reviewed. 

Each  slide  contained  both  a visual  representation  of  the  situation  and 
written  data  to  define  the  weapon  type,  the  actual  range  and  the  exposure  time 
involved.  In  addition,  as  each  slide  was  projected  the  experimenter  read  the 
information  aloud. 

The  sequence  of  99  slides  was  introduced  by  another  sequence  of  10 
slides  of  the  same  type  which  represented  situations  appearing  at  random  in 
the  later  sequence.  The  purpose  of  this  introductory  sequence  was  to  provide 
practice  in  the  techniques  involved.  Thus,  the  data  for  these  slides  have  not 
been  included  in  analysis.  The  99  slide  sequence  was  ordered  randomly  with 
the  following  constraint:  levels  of  at  least  two  of  the  situation  factors  had  to 
change  between  one  slide  and  another.  This  random  sequencing  was  performed 
by  a special  computer  program  designed  specifically  for  the  task. 

Direct  interrogation  was  also  employed  to  determine  from  each  subject 
the  range  at  which  perceived  threat  first  fell  to  zero  for  each  combination  of 
weapon,  exposure  time  and  relative  heading.  Upon  completion  of  the  109  slide 
sequence  outlined  in  previous  paragraphs,  the  subjects  were  asked  to  retrieve 
the  handout  they  had  been  given  earlier  which  described  all  threat  situations 
included  In  the  design.  This  handout  had  been  used  to  determine  the  situation 
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which  was  considered  most  threatening.  The  subjects  were  instructed  to  care- 
fully consider  the  situations  and,  without  regard  to  perceived  threat  values  they 
had  given  earlier,  to  record  on  the  handout,  the  range  at  which  they  would  feel 
no  threat  for  each  combination  of  weapon,  exposure  time  and  relative  heading. 

Indifference  techniques  were  employed  to  determine  data  with  which  to 
estimate  F(T).  However,  direct  interrogation  was  still  the  underlying  method 
of  eliciting  a response.  The  method  consisted  of  the  presentation  of  a sequence 
of  9 slides,  each  identical  in  composition  to  those  described  earlier.  As  each 
slide  was  presented,  each  subject  was  asked  to  record  an  unexposed  trave’ 
time  which  he  would  be  willing  to  endure  in  order  to  avoid  the  situation  shown  on 
the  slide.  Again,  the  subjects  were  asked  to  record  their  responses  on  pre- 
punched computer  data  cards  and  to  prepare  their  answers  without  regard  to 
other  answers  given  previously.  The  slides  were  randomly  sequenced  by  the 
procedure  described  earlier. 

The  sequence  was  presented  after  a lengthy  introduction  was  given  in 
which  a hypothetical  tactical  scenario  was  described  in  detail.  It  had  been 
found  in  preliminary  testing  with  another  group  of  subjects  that  such  an  intro- 
duction was  required  before  the  subjects  could  offer  responses  of  the  type 
desired.  The  implications  of  this  finding  will  be  discussed  in  the  analysis 
section  of  this  chapter. 

With  data  for  estimating  Dj(R,  (j>,  T)  and  F(T),  all  that  remained  was  to 
determine:  (1)  vehicle  information;  (2)  speed  information,  and;  (3)  relative 
overall  threat  information.  Information  regarding  the  maximum  effective  range 
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perceived  by  a subject  could  be  obtained  from  the  data  collected  to  estimate 

First,  the  subjects  were  asked  to  specify:  (a)  the  speed  they  would 
employ  while  flying  at  knap-of-the-earth  flight  levels,  and  (b)  the  speed  they 
would  employ  were  they  to  conduct  an  attack  with  running  fire.  Previous  to 
this , the  subjects  were  briefed  to  insure  that  they  understood  the  mission 
context  in  which  the  speeds  would  be  employed.  Moreover,  they  were  asked 
to  assume  that  they  were  flying  the  type  of  helicopter  they  felt  to  be  most  appro- 
priate to  the  mission  so  long  as  it  was  one  in  which  they  had  been  checked  out. 
The  type  of  vehicle  they  assumed  for  the  mission  was  recorded  in  addition 
to  the  two  speeds.  Finally,  the  subjects  were  asked  to  comment  on  the  way 
in  which  speeds  might  vary  during  knap-of-the-earth  flight  and  during  a running- 
fire  attack.  This  information  was  collected  at  the  beginning  of  the  experimental 
session. 

At  the  end  of  the  session,  the  subjects  were  asked  to  consider  carefully 
the  enemy  weapons  that  had  been  described  to  them  and  to  which  they  had  pro- 
vided perceived  threat  responses.  The  subjects  were  then  asked  to  rank  the 
weapons  to  indicate  in  general  how  they  rated  the  weapons  in  terms  of  overall 
threat.  For  example,  each  subject  was  asked  to  select  the  one  weapon  out  of 
the  three  he  would  least  like  to  encounter  on  the  battlefield.  This  weapon  was 
ranked  first.  The  procedure  was  repeated  for  the  two  remaining  weapons. 

After  the  ranking,  the  subjects  were  asked  to  rate  the  less  threatening 
weapons  in  relation  to  the  weapon  ranked  first.  A direct  measurement  was 
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elicited  on  the  scale  (0,  100).  These  results  then  provided  the  relative  weapon 
rating  data  that  were  desired  for  Model  Three.  No  attempt  was  made  to  relate 
these  data  to  the  preceived  threat  data  obtained  earlier  although  such  an  attempt 
might  prove  interesting  and  informative. 

Appendix  B contains  a summary  description  of  all  media  employed  dur- 
ing the  Phase  I Experiment,  including  examples  of  the  slides,  the  computer  data 
cards,  representative  handout  material  and  a summary  text  of  the  important 
oral  presentations  made  by  the  experimenter  during  the  session. 

The  Subject  Group 

A group  of  qualified  decision-makers  was  selected  from  among  students 
in  the  Aviation  Armament  Division,  Department  of  Standards  and  Instructor 
Training,  U.  S.  Army  Aviation  School,  Fort  Rucker,  Alabama.  These  sub- 
jects were  being  trained  as  helicopter  instructors  and  all  had  comparable  train- 
ing and  experience.  In  particular,  each  had  served  as  a gunship  pilot  in  Viet- 
nam, had  fired  weapons  in  combat  and  had  been  opposed  by  some  form  of  enemy 
antiaircraft  fire.  However,  as  a group  they  were  poorly  trained  in  the  doctrine 
of  helicopter  employment  in  a mid  intensity  warfare  environment.  ^ 

A summary  of  the  subject  group  qualifications  is  presented  in  Table  1. 

5 

As  may  be  noted,  the  group  consisted  of  eight  subjects. 


^Such  doctrine  is  only  now  being  developed  and  few  Army  personnel 
have  even  been  briefed  on  the  development  of  this  doctrine. 

,r>Tho  previous  experiment  summarized  in  Appendix  G was  conducted 
with  a group  of  live  subjects.  Thus,  the  subjects  in  Phase  1 were  numbered 
0 through  13. 
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observation  helicopter  in  Vietnam. 


Conduct  of  the  Experiment 

Arrangements  for  the  experiment,  including  assistance  in  location  of 
the  subject  group  was  provided  by  Major  Gary  O.  Lozier,  Office  of  Doctrine 
Development,  Literature  and  Plans,  U.  S.  Army  Aviation  School,  Fort  Rucker. 
His  aid  was  invaluable. 

A classroom  was  reserved  with  excellent  facilities  for  conducting  the 
experiment.  The  room  was  equipped  with  remote  controlled  lighting,  a screen 
for  viewing  slides,  a projector,  and  work  tables  of  sufficient  size  to  permit 
freedom  of  movement  for  the  subjects.  Adequate  room  was  available  between 
tables  to  insure  independent  responses. 

As  mentioned  in  Chapter  1 and  discussed  in  the  next  chapter,  the  exper- 
imental session  was  to  include  not  only  the  Phase  I Experiment  but  also  a por- 
tion of  the  Phase  II  Experiment.  This  Phase  II  activity  involved  the  selection 
of  routes  using  map  boards  and  required  extensive  supervision  on  the  part  of 
the  experimenter.  For  this  reason  and  to  prevent  undue  inconvenience  for  the 
subjects,  the  subject  group  was  divided  into  two  subgroups  of  four  subjects  each. 
The  first  subgroup  commenced  the  experimental  session  at  0800  hours  on  28 
December  1971  while  the  second  subgroup  was  scheduled  to  commence  at 
1000  hours.  However,  the  second  subgroup  was  prevented  from  starting  until 


As  each  subject  entered  the  room  he  was  randomly  assigned  a work 
table.  On  each  table  was  a numbered  supply  packet  containing  the  materials 
described  in  Appendix  B.  The  number  on  the  packet  became  the  subject  number 
with  which  he  would  be  identified  in  all  analyses . 

The  session  then  proceeded  with  an  introduction  in  which  the  overall 
purpose  of  the  experiment  was  explained.  Following  this  introduction,  the 
session  continued  as  outlined  in  the  previous  section.  No  problems  were 
encountered  as  all  the  experimental  procedures  had  been  perfected  in  the  pre- 
liminary experiment  outlined  in  Appendix  G. 

Analysis  of  Results 

Results  of  the  Phase  I Experiment  are  presented  in  detail  in  Appendix  C . 
The  purpose  of  this  section  is  to  describe  the  analysis  of  that  data,  to  present 
some  conclusions  that  were  made  regarding  the  use  of  the  data  as  input  to  the 
route  selection  model,  and  to  make  some  preliminary  observations  on  the 
apparent  decision  performance  of  helicopter  pilots. 

Helicopter  Types.  Comparison  of  Table  C.l  with  Table  1 indicates 
that  with  but  one  exception  (subject  6)  the  type  of  helicopter  selected  for  the 
hypothetical  mission  was  the  type  the  subject  had  flown  as  a gunship  in  Vietnam. 
Thus,  it  is  assumed  that  the  threat  perception  response  data  were  at  least 
gathered  in  an  environment  in  which  the  subjects  were  associating  with  a heli- 
copter with  which  they  were  thoroughly  familiar.  By  forcing  the  response  the 
subjects  were  encouraged  to  consider  the  threat  situations  in  this  manner. 
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Speeds.  Table  C.2  shows  that  both  knap -of -the -earth  flight  speeds  and 
attack  speeds  for  a particular  subject  are  in  general  highly  variable.  This  find- 
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ing  is  contrary  to  the  assumptions  of  the  route  selection  model  and  might  indicate 
an  area  in  which  enrichment  of  the  route  selection  model  would  be  beneficial. 
Another  result  apparent  in  Table  C.2  is  that  there  is  considerable  variation 
among  subjects.  Each  subject  seems  to  have  developed  his  own  personal 
style  through  experience . In  the  absence  of  a model  to  accept  variable  speed 
inputs,  the  median  value  for  knap -of -the -earth  flight  speed  was  selected  as  the 
value  to  be  used  during  Phase  n research.  However,  because  of  experimental 
design  requirements  of  Phase  II  (discussed  in  Chapter  4)  the  attack  speed 
selected  for  a subject  was  not  necessarily  the  median  value . One  speed  had 
to  be  selected  for  each  subject  pair  (6,  10),  (7,  11),  (8,  12)  and  (9,  13).  Thus, 
it  was  attempted  to  select  a speed  that  fell  in  the  range  of  speeds  common  to 
both  subjects  in  a particular  pair.  The  speeds  selected  for  Phase  II  research 
are  indicated  in  Table  C.2. 

Relative  Threat  Ratings.  The  data  in  Table  C . 3 can  be  used  to  compute 
the  relative  threat  variables  (C.)  needed  as  input  for  the  perceived  threat  model 
known  as  Model  Three.  A very  interesting  observation  begins  to  emerge  from 
examination  of  these  data.  Even  though  the  subjects  had  been  presented  the 
same  information  concerning  the  enemy  weapons , the  subjects  either  perceived 
the  information  differently  or  they  were  operating  on  a set  of  preconceived 
notions  developed  individually  or  perhaps  in  groups.  Three  of  the  six  possible 
orderings  of  the  weapons  are  present,  with  subjects  6,  7 and  10  giving  very 
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similar  results,  subjects  8 and  13  giving  similar  results  and  subjects  9,  11 
and  12  giving  similar  results.  Average  results,  when  normalized,  are  close 
to  those  given  by  subjects  9,  11  and  12. 

Asa  preliminary  observation  it  appears  that  the  pilot  population  may 
indeed  by  heterogeneous . It  was  not  expected  that  the  pilots  would  appear  so 
dissimilar  with  respect  to  the  simple  question  of  an  overall  rating  of  enemy 
weapons.  Nearly  uniform  behavior  similar  to  that  exhibited  by  subjects  6,  7 
and  10  or  by  subjects  9,  11,  and  12  had  been  expected.  Thus,  this  finding  served 
as  an  initial  indication  of  non-uniform  decision  behavior  among  the  subjects  and 
led  to  the  belief  that  further  evidence  of  such  behavior  would  become  apparent 
during  analyses  of  the  other  data  collected. 

Maximum  Effective  Range . The  other  set  of  input  data  required  for 
Model  Three,  the  perceived  maximum  effective  range  for  each  weapon  type,  can 
be  derived  from  the  maximum  threatening  range  data  collected  during  the  second 
factorial  experiment  associated  with  D. (R,  <}>  , T).  These  maximum  threatening 
range  data  appear  in  Table  C . 5 . It  may  be  seen  that  these  data  vary  strongly 
with  exposure  time,  they  are  only  moderately  variable  between  subjects,  and 
except  for  the  REDEYE  weapon,  they  are  almost  invariant  with  changes  in 
relative  heading.  There  is  some  indication  of  subject  inconsistency  when  the 
data  are  compared  with  perceived  threat  data  in  Tables  C.6  through  C.8.  How- 
ever, the  direct  interrogation  method  used  to  elicit  the  data  appears  to  have 
worked  fairly  well  in  that  overall  trends  in  the  data  appear  quite  reasonable. 

The  differences  in  data  trends  for  the  REDEYE  weapon  can  be  traced  to  the  fact 
that  this  weapon  is  much  more  effective  when  fired  at  the  rear  of  the  helicopter. 
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One  interesting  observation  to  be  made  is  that  for  the  longer  exposure 
times,  subjects  tend  to  consider  a weapon  threatening  at  ranges  in  excess  of 


I 
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published  estimates  of  weapon  effective  range . This  trend  indicates  that  the 
subjects  might  consider  a weapon  lethal  out  to  its  absolute  maximum  range 
capability.  Such  a trend  is  indicative  of  conservative  behavior  on  the  part  of 
the  pilot  population. 

Now,  the  data  required  for  each  subject  in  Model  Three  is  a function  of 
weapon  type  only.  Therefore,  an  average  was  taken  across  exposure  time  and 
relative  heading  to  obtain  the  desired  data.  Possibly  a more  representative 
approach  would  have  been  to  average  the  maximum  value  for  the  three  exposure 
times  across  relative  heading.  Also,  Model  Three  could  have  been  enriched  to 
allow  for  variation  in  maximum  threatening  range  as  a function  of  exposure 
time.  However,  neither  of  these  alternatives  was  adopted. 

Unexposed  Threat  Function.  The  data  in  Table  C .4  represent  values  of 

A A, 

the  factor  G(T)/T  where  G(T)  is  the  perceived  threat  value  recorded  during  the 

A A 

experiment  to  determine  data  to  estimate  Di(R,((>  , T),  T is  the  value  of  expo- 
sure time  that  was  involved  and  T is  the  value  of  unexposed  travel  time  the  pilot 
would  be  willing  to  endure  to  avoid  G(T).  This  factor  can  be  thought  of  as  the 
rate  of  accumulation  of  threat  while  unexposed. 

First,  it  appears  that  the  procedure  used  did  not  result  in  a very  con- 
sistent performance  by  the  subjects.  To  be  specific,  no  systematic  variation 
can  be  detected  as  the  factor  levels  change.  Furthermore,  it  is  not  apparent 
that  any  logical  reason  should  exist  to  explain  why  the  values  vary  as  they  do 
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as  the  factor  levels  of  the  equivalent  exposed  situation  change.  It  had  been 
hoped  that  at  least  for  a given  subject,  the  values  for  the  factor  would  be 
almost  constant.  This  would  result  in  a linear  form  for  F(T)  and  indicate 
additive  disutilities  at  least  between  any  two  unexposed  route  segments. 
However,  such  results  were  not  obtained.  Therefore,  it  is  proposed  that  the 
variability  is  strictly  an  indication  of  subject  inconsistency  in  the  assigned 

A 

task  as  it  was  presented  to  them.  Since  the  values  of  G(T)/T  are  quite  small 
in  relation  to  the  values  that  exist  for  G(T),  it  is  considered  to  be  unimportant 
that  an  accurate  functional  relationship  be  developed  for  F(T).  Thus,  the 
apparent  subject  inconsistency  is  judged  to  be  of  little  consequence.  Hence, 

a 

F(T)  is  assumed  linear  in  T for  each  subject  with  the  average  value  of  G(T)/T 
for  each  subject  being  used  as  the  constant  of  proportionality. 

Next,  the  reader  will  recall  that  the  subject  group  had  to  be  given  a 
detailed  description  of  the  overall  tactical  scenario  before  indifference  values 
could  be  elicited  with  which  to  estimate  F(T).  This  finding  indicates  that  the 
magnitude  of  the  range  of  F(T)  might  be  a strong  function  of  the  overall  battle- 
field situation  in  which  a route  is  to  be  planned.  Thus,  the  data  collected  dur- 
ing Phase  I are  definitely  applicable  only  to  the  class  of  route  selection  prob- 
lems studied  in  this  research.  However,  it  is  difficult  to  perceive  of 
militiary  route  selection  problems  involving  exposure  to  the  enemy  in  which  the 
magnitude  of  the  range  of  F(T)  would  approach  in  value  the  magnitude  of  the 
range  of  D.(R,  T).  Thus  in  future  research,  it  would  seem  that  collection 

of  data  to  estimate  F(T)  could  be  avoided.  In  fact,  it  is  suggested  that  the 


100 


I 

I 

I 


statement  of  the  Acid  Test  Model  relation  be  revised  as  follows: 
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• D^R,  4>,  T)  if  a segment  is  exposed  to  at 
least  one  enemy  weapon 

if  a segment  is  unexposed 


where  K is  simply  some  small  fraction  (.001)  of  the  magnitude  of  the  range  of 
Dj(R,  4>,  T)  and  all  other  variables  are  as  defined  on  page  73  of  this  chapter. 

Perceived  Threat.  Perceived  threat  data  appear  in  Tables  C.6  through 
C.8.  The  associated  maximum  threatening  range  data  discussed  earlier  appear 
in  Table  C . 5 . The  only  observation  that  may  be  made  about  the  data  in  this 
form  is  that  the  direct  interrogation  method  apparently  worked  fairly  well  in 
that  overall  trends  appear  quite  reasonable.  However,  some  evidence  of  inconsis- 
tent subject  performance  does  exist. 

Now,  the  reader  will  recall  that  it  is  assumed  that  the  jth  perceived 
threat  response  obtained  from  a given  subject  and  corresponding  to  a particular 
set  of  values  for  i , R , <}>  and  T can  be  expressed  as 


Oj=Di(R,  4>,  T)  + €j 

where  Ck  is  the  response  obtained  and  is  a normally  and  independently  dis- 
tributed random  error  with  zero  mean  and  variance  <r2.  With  this  definition  of 
the  underlying  phenomenon,  the  objective  of  analyses  of  the  perceived  threat 
data  was  to  obtain  a statistically  accurate  and  precise  estimate  of  Dj(R,  4>»  T). 
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However,  since  so  little  was  known  about  the  function,  it  was  decided  not  to 
postulate  an  a priori  functional  form  that  might  be  tested  through  regression 
analyses.  Instead,  it  was  anticipated  that  considerable  information  about  the 
function  could  first  be  determined  through  analysis  of  variance  techniques. 

Possibly  after  such  an  analysis,  regression  analyses  could  be  more  fruitfully 
applied. 

As  a start,  the  most  general  model  possible  was  assumed.  That  is,  the 
assumed  model  included  an  overall  mean,  all  main  effects  (subject,  weapon, 
exposure  time,  range  > relative  heading),  all  two-,  three-  and  four-factor  inter- 
actions and  an  error  term.  However,  this  approach  required  that  the  five-factor 
interaction  be  assumed  zero  in  order  to  obtain  an  estimate  of  the  error  term. 

Had  the  overall  factorial  been  replicated,  such  an  assumption  would  not  have 
been  required. 

Now,  as  mentioned  in  a previous  section,  the  design  used  to  obtain  the 
data  presents  some  problems  if  one  wishes  to  perform  exact  statistical  analyses 
of  the  data.  The  design  was  inherently  unbalanced  since  no  variation  in  rela- 
tive heading  was  considered  for  the  first  level  of  separation  range  (R  = 0).  That 
is,  only  one  replication  at  R = 0 was  performed  when  in  fact,  five  replications 
should  have  been  performed. 

One  approach  to  solving  the  problem  would  be  to  analyze  only  the  two 
intermediate  levels  of  R (40%  and  80%  maximum  effective  range).  However, 
this  approach  presents  a dilemma  since  only  a linear  effect  in  range  could  be 
determined;  and  as  stated  earlier,  it  was  desired  to  obtain  estimates  of  at 

102 


w 


I 

I 

| least  a quadratic  effect  for  all  factors.  Furthermore,  an  appreciable  amount 

of  the  information  available  would  not  be  used.  Therefore,  for  the  sake  of 
simplicity  and  in  view  of  the  desire  to  obtain  statistical  insight  with  regard  to 
the  overall  shape  of  the  response  surface,  the  analysis  performed  was  approx- 
imate. First,  the  available  data  set  was  modified  so  that  the  one  value  of  per- 
ceived threat  that  existed  at  R = 0 for  each  subject,  exposure  time  and  weapon 
type  was  repeated  four  times.  In  other  words,  the  data  were  analyzed  as 
though  a complete  factorial  experiment  had  been  conducted  in  which  the  five 
replications  required  at  R = 0 had  been  obtained.  Unfortunately,  this  step 
results  in  errored  estimates  of  all  quantities  in  the  model,  but  the  error  is 
hopefully  small.  For  example,  Table  2 shows  the  contribution  of  possibly 
erroneous  readings  in  the  computation  of  marginal  means  for  each  main  effect, 
expressed  as  the  absolute  number  of  possibly  erroneous  readings  and  as  a per- 
centage of  the  readings  comprising  each  mean.  It  is  seen  that  for  most  effects, 
observations  that  are  possibly  erroneous  account  for  20%  of  the  marginal  mean 
observations.  However,  the  marginal  means  for  relative  heading  and 
separation  range  consist  on  the  average  of  27%  errored  observations.  More- 
over, 80%  of  the  observations  comprising  the  marginal  mean  associated  with  the 
first  level  of  range  are  possibly  in  error.  Unfortunately,  no  data  were  gathered 
to  indicate  how  much  intrapersonal  variation  might  exist  since  no  replication 
was  performed.  If  intrapersonal  variation  is  small,  then  the  analysis  is  rela- 
tively free  of  error. 


Contribution  of  Possibly  Erroneous  Observations 


Marginal  Mean 

Number  of 

Errored  Observations 

Total  Number 
of  Observations 

% 

Subject 

36 

135 

20 

Weapon 

96 

360 

20 

Exposure  Time 

96 

360 

20 

Separation  Range 

Level  1 

288 

360 

80 

Level  2,  3 

0 

360 

0 

Average 

96 

360 

27 

Relative  Heading 

57.6 

216 

27 

Average 

A saving  factor  does  exist,  however.  The  errors  of  which  we  are  con- 
cerned affect  only  the  determination  of  which  factors  and  interactions  are 
statistically  significant.  If  certain  factors  and  interactions  appear  overwhelm- 
ingly significant  in  an  analysis  of  variance  it  might  be  assumed  that  observations 
without  error  would  not  affect  conclusions  to  be  drawn.  Plots  of  the  marginal 
means  would  still  indicate  data  trends  giving  rise  to  the  variation. 

With  all  the  cited  approximations  in  mind  we  present  at  Figure  7 an 
analysis  of  variance  table  for  the  perceived  threat  experiment.  It  is  readily 
apparent  from  this  type  of  analysis  that:  (1)  with  the  exception  of  the  weapon 


I 

I 


fc. 


I 

I 

I 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected  Mean  Signif- 

Variation 

Freedom 

Squares 

Squares 

Squares  icance 

1-Separation 

2 

917,618 

458,809 

4 

+ 360df  + 45df3 

Range 
2- Weapon 

2 

848,813 

424, 066 

d2 

e 

+ 360d|  + 45d|3 

Type 

3-Subject 

7 

7,573,518 

1,081,931 

q2 

e 

+ 135^2  *** 

3 

4- Exposure 

2 108,724,506 

54,362,240 

°2 

+ 360d2+45C^4  *** 

Time 

5-Relative 

4 

1,497,850 

374,463 

+ 216d?  + 27o 2 *** 

5 o o 

Heading 

12 

4 

1,611, 115 

402,779 

d2 

e 

- i5o*123  *** 

13 

14 

3,757,640 

268,4^3 

«J 

+ 45d23  *** 

14 

4 

1,597,581 

399,395 

+ 12^  15d2134  * 

15 

8 

817,075 

102, 134 

e 

+ 79d2  + qfl2  *** 

7^°15  y 135 

23 

14 

6,022,768 

430,198 

d2 

+ 45 cf2  *** 

e 

23 

24 

4 

655, 243 

163,811 

d2 

e 

+ 121<^4+  I5°234 

25 

8 

2,884,528 

360,566 

d2 

e 

+ 22°25+  *** 

34 

14 

5,221,514 

372,965 

d2 

e 

2 

+ 450  34  *** 

35 

28 

2, 05r>,  ^61 

73,216 

d2 

e 

+ 27 d2  *** 

35 

45 

8 

841,078 

105,135 

°2 

+ 72045+  *** 

123 

28 

1,042,468 

37, 23n 

o2 

e 

+ I5d2  *** 

123 

2 

2 2 

124 

8 

1,489,732 

186,216 

d 

e 

+ 40o  + 5d  J *** 
124  1234 

125 

16 

1,750,701 

109,419 

d2 

+ 2402  + 30  2 *** 

e 

125  1235 

Figure  7.  — Analysis  of  Variance  for  the  Perceived  Threat  Experiment 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected  Mean  Signif- 
Squares  icance 

134 

28 

4,299,961 

153,570 

°l  * 150m 

*** 

135 

56 

1,559,694 

27,85  2 

°e  * 90?35 

*** 

145 

16 

661,969 

41,373 

°e  + 240?45  * 30?345 

*** 

234 

28 

4,636,343 

165,584 

a2  + 15a 2 
e 234 

*** 

235 

56 

1,620,486 

28,937 

°e * 90 235 

*** 

245 

16 

654,190 

40,887 

°e  + * 302345 

*** 

345 

56 

1,156,340 

20,649 

°f  + S0345 

** 

1234 

56 

2,392,846 

42,729 

O2  + 5 o2 
e 1234 

*** 

1235 

112 

2,152,489 

19,219 

02  + 302 
e 1235 

** 

1245 

32 

950,913 

29,716 

a2  + 8d^ 
e 1245 

*** 

1345 

112 

1,691,697 

15,104 

02  + 302 
e J 1345 

* 

2345 

112 

1,947,036 

17,384 

a2  + 3cj2 

e ^2345 

* 

Residual 

224 

3,064,944 

13,683 

o2 

e 

Total 

1079  176,091,808 

Figure  7. — Analysis  of  Variance  for  the  Perceived  Threat  Experiment 

(continued) 

Note:*  Significant  at  a =0.10 
**  Significant  at  a =0.05 
***  Significant  at  a =0.01 
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and  separation  range  factors,  every  other  factor  considered  in  the  experiment 

t 

was  indeed  important;  and  (2)  a very  complicated  set  of  interactions  exists 
among  all  factors  studied. 

At  first  glance,  the  lack  of  a weapon  effect  is  very  surprising  since  it 
had  been  expected  that  the  type  of  weapon  being  considered  would  strongly 
influence  the  pilots'  perception  of  threat.  The  negative  result  can  be  explained, 
however.  It  was  noted  earlier  in  the  discussion  of  the  overall  enemy  weapon 
rating  data  obtained  for  Model  Three  that  the  pilots  differ  widely  in  their  rela- 
tive rankings  of  the  weapons.  This  phenomenon  is  further  illustrated  in  Figure 
7 where  it  may  be  noted  that  there  is  a very  strong  interaction  between  the  sub- 
ject factor  and  the  weapon  factor.  Thus,  it  is  possible  that  an  analysis  of 
variance  performed  for  each  individual  pilot  might  reveal  that  a weapon  effect 
does  exist  for  at  least  some  of  the  pilots.  In  fact,  such  an  analysis  was  actually 
performed  and  it  was  found  that  all  eight  subjects  did  register  a weapon  effect. 

The  lack  of  a separation  range  effect  is  also  explainable.  The  reader 
will  recall  that  the  maximum  separation  range  considered  in  the  experiment 
was  only  80%  of  the  published  maximum  effective  range  of  the  three  types  of 
enemy  weapons . It  is  entirely  possible  that  the  pilots  consider  the  weapons  to 
be  almost  uniformly  threatening  out  to  a range  in  excess  of  this  80%  value. 
Indeed,  as  was  discussed  previously,  an  analysis  of  maximum  threatening  range 
data  (Table  C.5)  indicates  that  under  certain  conditions  the  subjects  consider 
the  weapons  threatening  to  ranges  in  excess  of  the  maximum  effective  range  itself. 
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Table  3 shows  the  relationships  that  exist  between  the  maximum  ranges  used  in 
the  experiment  and  the  marginal  means  computed  from  the  maximum  threaten- 


ing range  data. 

The  reader  should  note  that  the  lack  of  a separation  range  effect  lends  at 
least  partial  support  to  Model  Three  as  a perceived  threat  model.  In  this  model, 
an  enemy  weapon  is  considered  threatening  if  it  is  within  the  maximum  effective 
range  assigned  to  it  by  the  decision  maker  . 


Table  3 

Maximum  Threatening  Range  Relationships 


Weapon  Type 

14.5  mm 

23  mm 

REDEYE 

Maximum  Range  Used 

1200 

2400 

2800 

Marginal  Mean  for  T = 5 seconds 

432 

918 

1277 

Marginal  Mean  for  T = 15  seconds 

1264 

2812 

2829 

Marginal  Mean  for  T = 45  seconds 

1570 

3018 

3249 

Examination  of  the  sums  of  squares  column  in  Figure  7 reveals  that 
exposure  time  is  by  far  the  most  important  factor  as  had  been  anticipated.  In 
fact,  over  60%  of  the  total  variability  can  be  attributed  to  this  factor.  However, 
the  fact  that  approximately  40%  of  the  total  variability  remains  unaccounted  for 
indicates  that  the  Acid  Test  model  should  still  consider  factors  other  than 
T alone. 


Another  interesting  observation  is  that  almost  all  of  the  exposure  time 


I 
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variability  can  be  attributed  to  a linear  effect.  Specifically,  by  the  use  of 
^ orthogonal  polynomials,  it  was  determined  that  98%  of  the  variation  is  explain- 

able as  a linear  effect.  This  finding  lends  support  to  the  assumption  of  a linear 
relationship  between  exposure  time  and  perceived  threat  in  the  Acid  Test  Model. 
Moreover,  the  linear  relationship  assumed  in  Model  One  and  Model  Three  are 
apparently  quite  reasonable. 

The  fact  that  there  is  a relatively  strong  subject  effect  lends  more  cre- 
dence to  the  proposition  that  a considerable  overall  difference  exists  between 
pilots,  at  least  with  respect  to  their  perception  of  threat.  However,  a more 
important  result  is  that  some  evidence  exists  to  suggest  that  the  pilots  apparently 
differ  even  with  respect  to  the  way  in  which  they  view  the  dependence  of  perceived 
threat  on  the  situation  variables.  For  example,  approximately  10%  of  the  total 
variability  can  be  attributed  to  two  factor  interactions  involving  the  subject 
factor.  In  comparison,  the  set  of  weapon  two-factor  interactions  contributes 
only  6%,  of  which  the  subject -weapon  interaction  is  over  half.  The  other  sets 
of  two-factor  interactions  contribute  less  than  5%  each,  with  the  subject-factor 
interaction  contributing  about  50%  to  each  set  on  the  average. 

Examination  of  the  residual  indicates  that  less  than  2%  of  the  total  var- 
iability is  unaccounted  for  by  the  effects  and  interactions  considered.  This 
result  indicates  that  the  model  is  very  good  at  representing  the  response  surface. 
However,  examination  of  the  list  of  significant  factors  and  interactions  indicates 
that  the  response  surface  is  an  extremely  complex  function  of  the  independent 
variables.  Thus,  it  would  lx-  very  difficult  to  propose  a functional  form  for 
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Dj(R,  <j>,  T)  to  be  studied  with  regression  analyses.  Hence,  the  decision  was 
made  that  the  data  for  each  subject  would  be  used  in  raw  form  as  input  to  the 
route  selection  model  in  order  to  permit  studies  involving  the  Acid  Test  threat 
perception  model.  A procedure  would  be  incorporated  into  the  route  selection 
model  to  linearly  interpolate  between  data  points  in  order  to  define  the  per- 
ceived threat  (DIJMN)  for  any  route  segment.  Considering  this  result,  all  the 
attention  paid  previously  to  the  approximations  involved  in  analysis  of  the  threat 
perception  data  was  certainly  important  for  understanding  but  of  little  conse- 
quence in  the  final  analysis. 

Additive  Disutilities.  One  result  of  the  findings  in  the  previous  para- 
graphs is  that  if  a decision  maker  did  in  fact  perceive  threat  as  predicted  by 
the  Acid  Test  Model,  then  the  additivity  assumption  of  Chapter  2 would  be  vio- 
lated. Not  only  do  complex  interactions  exist  among  the  effects  of  the  geo- 
metric variables , but  the  effects  of  the  individual  factors  themselves  exclude 
the  possibility  of  additive  segment  disutilities  to  compute  route  disutility.  For 
example , there  is  a fairly  strong  dependence  upon  the  variable  <J>  as  indicated  by 
its  significance.  Moreover,  by  the  use  of  orthogonal  polynomials,  it  has  been 
determined  that  94%  of  the  variability  can  be  attributed  to  a linear  effect.  Now, 
as  indicated  previously  there  is  also  a strong  linear  effect  in  T but  no  R effect. 
Even  if  there  were  no  interactions  among  these  effects,  additive  disutilities 
could  not  exist. 
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As  an  illustration  of  this  observation,  suppose  the  relation  for  perceived 
threat  were  simply 

D = a<j>  + bT 

where  a and  b are  constants  that  might  be  determined  through  regression  analyses. 
Now  suppose,  as  in  Figure  8,  a helicopter  flying  at  a speed  of  80  knots  were 
approaching  a route  segment,  the  execution  of  which  would  result  in  a total 
exposure  time  of  45  seconds.  As  illustrated,  the  midpoint  of  the  segment  is 
600  meters  from  a 14.5  mm  weapon  located  directly  abeam  of  the  segment. 

Direct  evaluation  of  perceived  threat  for  this  situation  yeilds  the  result 


D = ir  /2  a + 45  b # 

Now,  instead  of  direct  evaluation,  suppose  that  the  segment  were  divided 
into  three  subsegments  with  each  subsegment  representing  15  seconds  exposure 
as  illustrated.  If  valuewise  independence  exists  then  the  segmented  evaluation 
should  yield  the  same  results  as  the  direct  evaluation  scheme.  However,  such 
is  not  the  case.  The  result  is  approximately 

D = 3tt/2  a + 45  b . 

As  can  been  seen  the  only  way  in  which  additive  disutilities  could  exist 
would  be  for  the  constant  a to  be  zero,  a result  contrary  to  the  findings  of 
the  analysis  of  variance. 

As  an  additional  check  on  the  above  result,  an  analysis  of  the  type  out- 
lined was  performed  using-  actual  perceived  threat  data  from  Tables  C.6  through 
(’.8.  The  direct  evaluation  scheme  employed  45  second  exposure  time  duta 
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Figure  8.  — Schematic  of  Threat  Evaluation  Alternatives 


Table  4 


Comparison  of  Threat  Evaluation  Results 


Subject 

6 

7 

8 
9 

10 

11 

12 


13 


Direct  Evaluation 

Segmented  Evaluation 

Percent  Difference 

900 

1917 

113 

500 

1384 

177 

800 

1313 

65 

985 

2233 

127 

1000 

2133 

113 

900 

1251 

39 

900 

1609 

79 

975 

1706 

75 

870 

1693 

95 
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directly  from  the  tables , and  the  results  appear  in  the  column  labeled  "direct 
evaluation"  in  Table  4.  Data  for  15  second  exposure  time  were  interpolated 
from  the  tables  with  the  final  results  appearing  in  the  column  of  Table  4 labeled 
"segmented  evaluation."  As  can  be  seen  the  two  sets  of  results  are  quite  dif- 
ferent, with  differences  of  more  than  100%  very  common.  Such  results  are  in 
agreement  with  those  of  the  simple  analysis  cited  earlier. 

The  implications  of  these  findings  are  discussed  in  more  detail  in  Chap- 
ters 4 and  6.  However,  the  principal  implication  is  that  the  route  selection 
model  can  only  approximately  select  routes  for  decision-makers  who  do  indeed 
perceive  threat  as  described  by  the  Acid  Test  Model.  To  select  routes  exactly 
would  require  that  the  additivity  assumption  be  dropped.  This  would  involve  a 
modification  of  the  route  selection  model  that  is  beyond  the  scope  of  the 
present  research. 


CHAPTER  4 


PHASE  II  EXPERIMENTATION 
by  Don  C.  Hutcherson 

Introduction 

In  this  chapter  a comprehensive  description  of  the  Phase  II  Experiment  is 
presented.  The  reader  will  recall  that  the  overall  purpose  of  this  experiment  was 
to  test  the  predictive  qualities  of  the  route  planning  model  derived  in  Chapter  2. 
More  precisely,  the  experiment  was  intended  to  determine  how  well  a route  pre- 
dicted by  the  model  might  fare  in  comparison  with  a route  that  might  actually  be 
planned  by  a decision  maker,  given  a particular  tactical  situation.  In  addition, 
the  experiment  was  intended  to  determine  which  of  the  three  threat  perception 
models  proposed  in  Chapter  3 is  most  appropriate  for  use  in  the  route  planning 
model.  Finally,  it  was  desired  that  the  Phase  II  Experiment  should  yield  more 
information  about  the  decision  processes  employed  by  a pilot  in  the  route  plan- 
ning task  and  about  differences  that  might  exist  among  pilots  in  this  decision 
task. 

One  of  the  first  and  most  critical  research  tasks  associated  with  Phase  II 
was  development  of  methods  to  be  used  in  measuring  the  "goodness"  of  routes 
predicted  by  the  route  planning  model.  Without  such  methods  it  would  be  dif- 
ficult to  determine  how  well  such  routes  fare  in  comparison  to  routes  actually 
selected  by  decision  makers.  Moreover,  it  would  be  difficult  to  determine  which 
of  the  three  threat  perception  models  should  be  used  in  the  route  selection  model. 
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The  second  major  task  associated  with  Phase  II  was  development  of  an 
experimental  design  to  be  used  in  gathering  data.  The  design  had  to  yield  the 
required  information  efficiently  and  could  not  be  too  extensive  since  human 
subjects  were  being  used. 

The  final  Phase  II  research  task  was  associated  with  analysis  of  data 
gathered  during  the  experiment.  During  this  task,  conclusions  were  formulated 
with  respect  to  the  performance  of  the  route  selection  model  in  general  and  the 
three  threat  perception  models  in  particular.  In  addition,  further  conclusions 
about  pilot  decision  making  behavior  in  the  route  planning  task  were  reached. 

All  the  topics  discussed  above  are  described  at  length  in  the  remaining 
sections  of  this  chapter.  First,  the  method  selected  to  measure  the  "goodness" 
of  a route  predicted  by  the  planning  model  will  be  discussed  along  with  the  tech- 
niques employed  to  obtain  such  measures.  Then,  the  experimental  design  de- 
veloped to  gather  the  required  data  is  discussed.  Finally,  conduct  of  the  exper- 
iment and  analyses  of  the  results  are  presented  as  separate  topics. 

Validation  Measure 

As  stated  previously,  development  of  methods  to  be  used  in  validating 
the  predictive  qualities  of  the  route  selection  model  was  one  of  the  most  crucial 
activities  performed  during  the  Phase  II  portion  of  the  research  program. 

Since  the  route  selection  model  had  been  constructed  in  response  to  a hueristic 
theory  whose  individual  elements  had  not  been  tested  as  part  of  the  research 
program,  it  was  critically  important  that  a good  method  be  developed  to  deter- 
mine how  the  model  performed  as  a whole. 

lift 
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From  the  very  beginning  it  was  clear  that  the  predictive  qualities  of  the 
model  could  not  be  evaluated  objectively.  Given  that  the  model  was  intended  to 
predict  the  routes  that  individual  pilots  would  select  when  faced  with  specific 
tactical  situations , it  was  obvious  that  the  performance  of  the  model  could  be 
judged  only  by  those  for  whom  the  predictions  were  intended.  Moreover,  it  was 
anticipated  that  there  would  be  no  objective  basis  from  which  these  judgments  could 
be  rendered.  For  example,  it  is  highly  unlikely  the  prediction  model  would  duplicate 
exactly  the  route  chosen  by  an  individual  pilot.  Because  of  the  limitations  imposed  by 
the  method  used  to  construct  route  alternatives  in  the  model,  a subject  might  choose  a 
route  which  is  not  part  of  the  hypothesized  set  of  alternatives  but  which  is  close  to 
several  members  of  this  set.  Thus , the  predicted  route  would  almost  always 
differ  in  some  respect  from  the  route  the  subject  had  chosen.  A diverse  set  of 
objective  similarity  measures  could  be  developed  to  describe  these  differences, 
but  the  relative  importance  of  each  measure  would  remain  a matter  of  judgment. 

It  was  decided  then,  that  the  experimental  subjects  would  be  asked  to 
develop  a route  plan  for  each  of  several  tactical  situations.  Following  this  they 
would  be  asked  to  make  direct  comparisons  between  the  routes  they  chose  and 
the  corresponding  predicted  routes  developed  by  the  route  selection  model. 

The  direct  interrogation  method  had  been  used  successfully  during  Phase  I and 
was  felt  to  be  at  least  as  applicable  to  the  Phase  II  Experiment.  In  fact,  the 
Phase  II  task  was  almost  identical  to  some  of  the  tasks  performed  by  subjects 
in  experiments  conducted  by  Lawrence  (1968).  In  his  studies,  alternative  methods 
of  preparing  armored  combat  battle  plans  were  subjectively  ranked  and  rated  by 


the  subjects,  and  the  procedure  proved  to  be  quite  successful. 
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One  approach  that  was  tried  in  the  preliminary  experiment  (Appendix  G) 
was  to  have  each  subject  rank  a set  of  routes  that  included  the  route  he  had  chosen 
and  several  alternative  routes  developed  by  the  route  selection  model . The  route 
considered  to  be  the  most  preferred  for  the  specified  situation  was  ranked  first 
by  each  subject.  Then  each  subject  compared  the  remaining  routes  and  ranked 
the  best  one  remaining  second.  The  process  continued  until  all  route  alternatives 
had  been  ranked.  As  the  routes  were  ranked,  they  were  also  rated  on  a scale  of 
zero  to  one  hundred  to  indicate  the  subjective  worth  of  the  alternative  routes  rela- 
tive to  the  route  ranked  first.  The  route  ranked  first  was  given  a rating  of  one 
hundred. 

While  the  above  procedure  produced  reasonable  results  and  was  easily 
executed  by  the  subjects , it  was  felt  that  the  procedure  would  not  be  acceptable 
for  application  during  the  Phase  II  Experiment.  The  experimental  design  employed 
(to  be  discussed)  resulted  in  a very  large  number  of  alternatives  for  each  route 
selection  situation.  It  was  feared  that  the  subjects  would  find  it  difficult  to 
examine  such  a large  set  of  alternatives  simultaneously  and  render  meaningful 
judgments.  Therefore,  the  following  procedure  was  used.  Each  subject  was 
asked  to  perform  a series  of  comparisons  between  only  two  alternatives . In  each 
case,  one  alternative  was  the  route  he  had  planned  himself  and  the  other  alterna- 
tive was  one  of  the  routes  predicted  by  the  route  selection  model . The  subject 
indicated  which  route  he  preferred  and  the  rating  that  the  "non-preferred"  route 
would  receive  on  a scale  of  zero  to  one  hundred.  Moreover,  the  regions  of  the 
rating  scale  were  given  definite  qualitative  meanings  to  permit  the  subjects  to 

associate  a rating  with  their  feelings  about  a pair  of  route  alternatives. 
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The  association  was  as  follows : 


100:  the  two  routes  are  equally  preferable, 

95-99:  the  two  routes  are  judged  to  be  almost  identical, 

70-94:  one  route  is  mildly  preferred  to  the  other,  and 
0-69:  one  route  is  strongly  perferred  to  the  other. 

The  subjects  were  told  that  should  they  judge  a pair  of  routes  to  be  in  the 

i 

first  category  above,  that  they  should  have  to  flip  a coin  to  determine  which 
route  would  be  followed.  If  a pair  of  routes  were  judged  to  be  in  the  second 
category  then  they  should  be  so  close  to  one  another  in  preference  that  no  specific 
reason  for  selecting  one  roui.e  over  the  other  could  be  cited,  although  a prefer- 
ence would  exist.  Finally,  the  subjects  were  told  that  the  last  two  categories 
should  contain  routes  for  which  specific  reasons  for  preference  could  be  cited. 
Whether  mild  preference  or  strong  perference  exists  would  be  merely  a matter 
of  degree.  If  a major  criterion  used  by  a subject  for  route  selection  were  fla- 
grantly violated  by  a particular  alternative,  strong  preference  should  exist  for 
the  other  alternative. 

The  method  outlined  above  has  one  advantage  in  that  it  results  in  fairly 
easy  tasks  for  a subject  to  perform.  The  concepts  involved  are  also  simple 
enough  to  permit  quick  understanding  by  the  subjects.  Finally,  the  rating  given 
to  a particular  route  alternative  takes  on  a definite  qualitative  meaning  so  that 
the  relative  "goodness"  of  a route  may  be  inferred  from  its  rating.  There  are 
definite  disadvantages  to  the  method,  however.  First,  the  procedure  is  not  suf- 


ficient to  construct  a complete  ordinal  scale.  Next,  a test  for  intransitivity  as 
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exhibitedby  the  presence  of  circular  triads  for  example,  is  not  available.  Finally, 
there  is  the  possibility  of  bias  in  favor  of  (or  against)  the  route  the  decision- 
maker planned  himself  since  he  always  rates  route  alternatives  against  the 
route  he  knows  he  selected.  The  first  two  of  these  deficiencies  would  have  re- 
quired a tremendous  amount  of  labor  on  the  part  of  the  subjects  and  the  experimenter 
to  correct.  Unfortunately,  the  time  available  for  contact  with  the  subjects  was 
insufficient  for  a complete  ordinal  scaling.  The  last  deficiency  could  be  avoided 
only  if  some  method  were  available  to  conceal  the  identity  of  a subject's  per- 
sonal plan.  Such  methods  were  not  incorporated  in  the  Phase  II  experimental  plan. 

Experimental  Design 

Characteristics  of  the  design  developed  for  the  Phase  II  Experiment 
can  best  be  understood  by  an  analysis  of  the  types  of  information  that  were  de- 
sired as  output.  First,  of  course,  was  the  desire  to  determine  which  model 
(or  models)  of  perceived  threat  was  the  best  to  be  used  in  the  route  selection 
model.  This  suggested  that  the  qualitative  factor  M should  be  included  in  the 
design,  where  M stands  for  type  of  perceived  threat  model.  The  effect  of  this 
factor  would  be  fixed,  and  the  number  of  levels  m would  be  three. 

Second,  there  was  the  desire  to  determine  how  the  model  performed  in 
a variety  of  tactical  situations.  Coupled  with  this,  there  was  a desire  to  deter- 
mine how  the  model  fared  as  a function  of  the  decision-maker  doing  the  evalua- 
tion of  its  results.  To  obtain  a true  subject  effect  it  was  obvious  that  at  least 
two  subjects  would  have  to  be  presented  with  the  same  tactical  situations.  In 
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order  to  study  as  many  tactical  situations  as  possible  with  a fairly  simple  de- 
sign and  still  measure  a true  subject  effect,  itwas  decided  to  form  subject  pairs 
who  would  be  presented  the  same  set  of  tactical  situations.  This  decision  worked 
out  well  since  the  subject  group  available  for  evaluating  the  route  selection  model 
results  was  constrained  to  be  the  group  from  which  input  data  had  been  collected 
during  Phase  I.  Of  course  this  meant  that  eight  subjects  were  available.  Thus, 
the  qualitative  subject  factor  S was  measured  at  s=2  levels.  Also,  since  the 
subjects  had  been  selected  at  random,  the  subject  effect  was  considered  ran- 
dom. The  subject  pairs  were  (6,  10),  (7,  11), (8,  12),  and  (9,  13). 

Now,  each  subject  pair  was  presented  with  a basically  different  tactical 
situation  in  which  routes  were  to  be  planned.  Each  situation  was  the  same  in 
that  the  battlefield  over  which  routes  were  desired  was  the  same,  and  two  enemy 
weapons  were  present  in  each  case.  However,  the  situations  were  different  in 
that  the  position  and  type  of  each  enemy  weapon  changed,  but  in  all  cases  the 
types  of  weapons  used  were  constrained  to  be  those  that  had  been  analyzed  during 
the  Phase  I Experiment.  A summary  of  the  tactical  situation  layouts  appears 
in  Appendixes  A and  D. 

Four  different  route  plans  were  prepared  by  each  subject  in  a subject  pair. 
Thus , the  qualitative  route  (or  situation)  factor  R was  measured  at  r = 4 levels . 
Moreover,  because  of  the  way  the  levels  were  selected,  R was  considered  to  pro- 
duce a fixed  effect.  In  general,  the  four  routes  differed  only  with  respect  to  the 
starting  points  specified  and  the  targets  to  be  attacked.  Routes  1 and  3 started 
from  the  left  (or  west)  side  of  the  battlefield  while  routes  2 and  4 started  from  the 
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top  (or  north)  side  of  the  battlefield.  The  starting  positions  for  routes  1 and  3 
were  not  intervisible  with  either  of  the  enemy  weapons  while  routes  2 and  4 
started  from  positions  that  were  intervisible  with  at  least  one  of  the  weapons.  In 
general  routes  1 and  2 were  to  be  prepared  in  such  a way  as  to  permit  an  attack 
upon  one  of  the  enemy  weapons  while  routes  3 and  4 were  directed  at  the  other 
weapon.  Detailed  information  concerning  the  four  routes  for  each  tactical 
situation  is  contained  in  Appendix  A and  Appendix  D. 

Next,  there  was  a desire  to  gather  information  about  the  decision  process 
by  which  a pilot  selects  running  fire  as  opposed  to  hover  fire.  From  Chapter  2 
the  reader  will  recall  that  these  two  attack  modes  differ  in  that  running  fire  employs 
a forward  speed  during  the  entire  attack  while  hover  fire  employs  a "pop  up"  to  a 
firing  position.  Even  though  a subject  was  allowed  to  select  either  type  of  fire  as 
part  of  his  route  plan,  the  route  selection  model  was  exercised  to  determine  the 
best  route  plan  for  both  attack  modes.  Thus,  the  qualitative  attack  mode  factor 
A was  measured  at  a = 2 levels  and  was  considered  to  produce  a fixed  effect. 

Finally,  a block  effect  was  introduced  as  follows.  It  was  desired  to  deter- 
mine whether  the  route  selection  model  would  be  evaluated  more  favorably  by  a 
decision  maker  if  threat  perception  data  used  in  the  route  selection  model  were 
in  fact  the  data  provided  by  him  during  Phase  I.  Thus,  one  set  of  route  alternatives 
were  prepared  in  which  the  input  data  used  were  those  collected  during  Phase  I. 
Another  set  of  alternatives  was  produced  in  which  the  input  data  used  were  those 
obtained  by  averaging  the  data  collected  from  all  subjects  during  Phase  I.  If  no 
data  set  block  effect  were  found  to  exist  and  if  the  block  effect  did  not  interact 
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with  the  subject  effect  then  it  could  be  assumed  that  the  subjects  came  from  a 
homogeneous  population  of  pilots.  Moreover,  it  could  be  assumed  that  the  route 
selection  model  could  operate  on  population  average  data  as  opposed  to  data  from 
individuals.  Thus,  the  data  set  block  factor  D was  measured  at  d = 2 levels  and 
was  considered  to  produce  a fixed  effect.  For  purposes  of  identification  the 
alternatives  produced  by  the  average  input  data  were  called  the  "super  subject" 
alternatives.  The  super  subject  was  essentially  a ninth  subject  in  the  experiment. 

From  all  previous  discussion  in  this  section,  it  can  be  seen  that  the  over- 
all Phase  II  experimental  design  consisted  of  four  independent  five-factor  com- 
plete factorial  experiments.  That  is,  the  design  consisted  of  four  (s)x(r)x(m)x(a)x(d) 
complete  factorials  where  the  symbols  are  as  defined  in  previous  paragraphs. 
Thus0  a total  of  96  observations  were  collected  in  each  factorial  for  a grand  total 
of  384  observations. 

The  model  assumed  for  analysis  purposes  included  all  main  effects  and 
all  two-factor  interactions,  with  all  three  factor  and  higher  order  interactions 
assumed  to  be  zero.  This  model  applied  to  each  of  the  four  factorial  experi- 
ments. Finally,  all  five  factors  were  qualitative,  and  four  out  of  the  five  were 
considered  to  produce  fixed  effects. 

Conduct  of  the  Experiment 

As  mentioned  in  Chapter  1 , the  first  step  of  the  Phase  II  Experiment  was 
actually  conducted  during  the  experimental  session  devoted  primarily  to  the 
Phase  I Experiment.  During  this  step  each  subject  was  asked  to  plan  the  four 
routes  discussed  in  the  previous  section.  As  a second  step  in  the  Phase  II 
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Experiment,  data  collected  during  Phase  I were  prepared  as  input  data  for  the 
route  alternatives  specified  by  the  experimental  design.  The  final  step  consisted 
of  an  experimental  session  in  which  the  subjects  were  confronted  with  the  route 
alternatives  and  asked  to  rank  and  rate  the  alternatives  in  the  manner  described 
earlier. 

Now,  because  some  reaction  time  was  necessary  between  the  Phase  I 
experimental  session  and  the  final  Phase  II  session  in  order  to  permit  execution 
of  the  experimental  design  with  the  route  selection  model,  the  possibility  of  an 
unwanted  block  effect  arose.  However,  it  was  impossible  to  eliminate  the  pos- 
sibility. It  was  desired  that  the  subjects  prepare  their  route  plans  while  the 
concepts  discussed  during  Phase  I were  still  fresh  in  their  minds  and  while  an 
intimate  knowledge  of  the  enemy  weapons  characteristics  still  existed.  If  the 
route  planning  exercise  had  been  postponed  until  the  route  alternatives  were 
available  these  benefits  would  have  been  lost. 

Step  1.  The  first  step  of  Phase  II  was  accomplished  at  Fort  Rucker, 
Alabama  on  December  28,  1971.  After  completion  of  the  Phase  I experimental 
session,  a brief  discussion  of  the  route  planning  task  was  presented.  The  sub- 
jects were  asked  to  recall  the  tactical  scenario  that  was  described  as  an  intro- 
duction to  the  unexposed  threat  exercise  of  Phase  I (see  Appendix  B) . Moreover , 
they  were  asked  to  plan  their  routes  as  if  this  scenario  applied  in  each  of  the 
tactical  situations  P>r  which  routes  were  desired. 

Next,  the  subjects  were  distributed  a brochure  describing  the  character- 
istics of  the  TOW  missile  system.  They  were  told  to  study  the  document  carefully 
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as  this  would  be  the  weapon  they  would  employ  during  execution  of  each  route. 
Furthermore,  they  were  told  that  the  planned  routes  could  utilize  either  running 
fire  or  hover  fire,  should  include  just  one  attack  upon  a target,  and  the  attack 


should  conform  to  the  capabilities  of  the  TOW  weapon  system. 

Next,  a composite  map  board  was  distributed  to  each  subject  showing 

the  boundaries  of  the  battlefield,  elevation  contour  lines,  boundaries  of  forested 

areas , locations  of  the  two  enemy  weapons  and  regions  in  which  intervisibility 

existed  with  each  of  the  weapons  from  a knap-of-the-earth  flight  altitude.  This 

last  feature  of  the  map  board  represented  a departure  from  reality  in  that  it  made 

more  information  available  to  the  subject  than  he  normally  has  in  combat.  How- 
1 

ever,  the  LOS  maps  were  included  to  make  up  for  two  factors.  First,  the  type 
of  contour  map  used  was  prepared  by  computer  and  was  a departure  from  the  type 
of  map  normally  used  by  a military  pilot.  Second,  the  pilot  would  normally  have 
some  familiarity  with  terrain  over  which  his  mission  is  to  be  conducted.  In  fact, 
a pilot  can  frequently  see  the  terrain  over  which  a route  is  to  be  flown  at  the  time 
he  plans  his  route. 

Next,  each  subject  was  distributed  a variety  of  materials  to  be  used  in 
drawing  routes.  These  included  grease  pencils  and  clear  acetate  overlays  for 
recording  routes  and  several  scales  to  be  used  in  measuring  time  and  distance. 


Line  of  sight  (sec  Chapter  2 and  Appendix  A). 
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Finally,  the  subjects  were  given  a set  of  preprinted  forms  describing  the 
details  of  each  route  to  be  planned  and  providing  answer  spaces  for  recording 
various  characteristics  of  the  routes  selected.  The  subjects  then  recorded  their 
routes  as  directed.  A representative  sample  of  the  materials  provided  each 
subject  is  presented  and  described  in  Appendix  D. 

Step  2.  The  second  step  of  Phase  II  consisted  of  preparing  data  col- 
lected during  Phase  I as  input  to  the  route  selection  model  and  then  exercising 
the  model  as  prescribed  by  the  experimental  design.  This 

task  required  approximately  four  weeks  to  complete,  with  another  two  weeks 
required  to  prepare  the  results  in  such  a way  that  they  could  be  presented  to  the 
subjects  for  evaluation. 

Each  time  the  route  selection  model  was  exercised  at  a given  combination 
of  factor  levels,  a deck  of  cards  was  produced  which  gave  the  coordinates  of  the 
end  points  of  each  segment  in  the  resultant  route.  These  decks  were  collected, 
collated  and  used  as  input  to  another  computer  program  that  generated  instruc- 
tions to  a two-dimensional  plotter.  In  turn,  the  plotter  produced  a translucent 
overlay  depicting  the  route. 

Now,  the  route  description  decks  were  collated  in  such  a way  that  a given 
overlay  would  depict  as  many  as  three  distinct  alternatives  for  a given  route 
selection  situation.  The  factors  S (subject),  A (attack  mode),  R (route/situation) 
and  D (data  set)  were  held  constant  for  each  overlay  with  the  only  variable  being 
M (perception  model).  The  resultant  alternatives  on  each  overlay  were  annotated 
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to  identify:  1)  the  direction  of  motion,  2)  the  point  at  which  the  missile  was 
launched,  and  3)  the  number  of  the  alternative.  In  no  way  could  a subject 
identify  the  perception  model  being  used  to  select  a given  alternative. 

In  the  course  of  preparing  the  alternative  overlays  it  was  discovered 
that  in  many  instances  two  different  combinations  of  factor  levels  would  produce 
the  same  route  alternative  for  a given  situation.  Thus,  instead  of  producing  a 
complete  factorial  layout  a design  in  which  certain  observations  were  missing 
was  being  produced.  That  is,  instead  of  a subject  being  presented  with  12  route 
alternatives  for  each  route  he  had  selected,  a fewer  number  of  distinct  alterna- 
tives was  being  produced.  The  ramifications  of  this  development  are  discus- 
sed in  the  analysis  section  to  follow. 

Step  3.  The  final  step  of  the  Phase  II  experimentation  plan  was  con- 
ducted at  Fort  Rucker  on  February  15,  1972,  The  subjects  were  divided  into 
two  groups  of  four  subjects  each  as  had  been  done  during  Phase  I experimenta- 
tion, with  the  first  group  commencing  its  session  at  0800  hours.  The  second 
group  was  scheduled  to  start  at  1000. 

Two  groups  were  desirable  to  permit  closer  supervision  of  the  subject 
activities.  Furthermore,  facilities  available  for  the  session  were  marginal  with 
room  barely  available  for  four  subjects.  Finally,  only  one  set  of  super  subject 
route  alternatives  had  been  prepared.  The  two  sessions  were  identical  in  content. 

After  preliminary  instructions,  the  subjects  were  handed  the  packets  of 
materials  described  in  detail  in  Appendix  D.  First,  the  subjects  transferred  the 
routes  they  had  prepared  during  Phase  I from  the  clear  acetate  they  had  used 
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earlier  to  the  opaque  situation  maps  contained  in  their  supply  packet.  This 
activity  had  two  purposes.  First,  it  served  to  refamiliarize  them  with  the  situ- 
ations they  had  faced  during  Phase  I and  the  routes  they  had  planned  as  a response. 
It  also  served  to  produce  a more  suitable  experimental  medium  to  be  used  with 
the  translucent  route  alternative  overlays  to  be  used  later. 

Next,  the  subjects  were  given  ranking  and  rating  instructions  corresponding 
to  those  described  in  an  earlier  section  of  this  chapter.  The  subjects  then  began 

the  ranking  and  rating  exercise,  and  they  all  appeared  to  be  enthusiastic  in  their 
efforts.  For  the  most  part  they  were  amazed  that  a computer  could  be  made  to 

perform  a decision  function  in  which  the  results  were  so  visible  and  so  dynamic. 

Some  subjects  were  very  conscientious  in  the  task  and  added  copious  remarks  to 

their  evaluation  forms  to  indicate  their  reasons  for  particular  rankings . 

During  the  two  sessions  it  was  discovered  that  two  of  the  subjects  had 

misinterpreted  instructions  during  Phase  I and  had  prepared  their  route  plans 
o 

erroneously  • The  subjects  were  asked  to  make  a note  of  their  misinterpretation 
on  their  answer  sheets  and  to  continue  by  using  their  erroneous  routes  in  an  unal- 
tered form.  Examination  of  their  response  data  later  led  to  the  conclusion  that 
their  data  were  biased  in  favor  of  the  route  selection  model  alternatives.  Con- 
sequently, their  data  were  dropped  from  consideration  in  all  later  analyses. 

The  ramifications  of  this  decision  are  discussed  in  the  analysis  section  to  follow. 


The  subjects  had  erroneously  assumed  that  a firing  position  could  not  be 
selected  over  terrain  that  was  indicated  to  provide  protected  routes  for  heli- 
copters flying  a knap-of-thc-carth  flight  profile.  They  failed  to  realize  that 
one  could  climb  to  a firing  position  from  anywhere  within  the  range  of  the 
TOW  weapon  system. 
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Analysis  of  Results 


An  obvious  first  step  in  the  analysis  of  results  was  to  perform  an  analysis 
of  variance  upon  the  data  obtained  from  the  Phase  n Experiment.  However,  be- 
cause of  developments  alluded  to  in  the  preceding  section  the  analysis  was  not  as 
straightforward  as  had  been  intended.  First,  because  data  for  subjects  9 and  10 
had  been  dropped,  there  were  only  two  of  the  five-factor  factorials  instead  of  the 
four  anticipated.  The  other  two  factorials  degenerated  into  four-factor  factorials 
with  no  subject  factor.  Second,  because  different  factor  level  combinations  did 
not  always  produce  unique  alternatives,  the  originally  intended  complete  factor- 
ials degenerated  into  incomplete  factorials.  As  an  example  of  an  extreme  case, 
subject  6 was  presented  only  five  alternatives  for  route  1 instead  of  a possible 
twelve  alternatives. 

Table  E.  1 shows  the  numbering  scheme  for  alternatives  as  presented  to 
the  subjects  during  Phase  II.  From  this  table  one  may  partially  infer  the  amount 
of  overall  degeneration  that  actually  occurred.  However,  degeneration  also 
occurred  between  subjects,  but  this  fact  is  not  revealed  in  Table  E.  1 because  of 
the  numbering  scheme  presented. 

Therefore,  an  interesting  problem  occurred  before  the  analysis  of  variance 
could  even  be  attempted.  Some  method  had  to  be  developed  to  account  for  the 
missing  data.  Also,  it  was  found  through  further  analyses  of  the  problem  that 
some  interesting  observations  could  be  made  concerning  the  performance  of  the 
model  without  even  examining  the  data  collected  during  Phase  n. 
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Analysis  of  Missing  Data  Phenomenon.  The  factors  that  can  produce  mis- 


sing data  are  limited  to  the  subject  factor  S (between  7 and  11  and  between  8 and 
12),  the  perception  model  factor  M (1-Model  One,  3 -Model Three , AT-Acid  Test 
Model),  and  the  data  set  factor  D (1-personal  data  set,  2-super  subject  data  set). 
The  attack  mode  factor  A (RF-running  fire,  HF-hover  fire)  and  the  route  factor 
R (1,2, 3, 4)  cannot  result  in  missing  data.  Table  E.2  shows  the  degree  to  which 
missing  data  occurs  as  a function  of  the  levels  of  D;  Table  E.3  gives  comparable 
results  for  M and  Table  E.4  shows  the  effect  of  S. 

Now,  analysis  of  Table  E.  2 shows  that  if  one  were  to  combine  all  subjects, 
a study  of  the  block  effect  D would  be  based  upon  data  of  which  only  55%  of  the 
observations  reveal  a true  block  effect.  This  fact  indicates  that  for  the  overall 
variations  that  existed  between  subjects  during  Phase  I,  the  route  selection  model 
is,  to  a degree,  insensitive.  However,  further  analysis  reveals  that  Model  One 
is  the  chief  culprit.  If  one  were  to  drop  Model  One  from  the  analysis,  then  only 
16%  of  the  observations  would  be  missing.  Thus,  without  Model  One,  the  route 
selection  model  can  be  considered  to  be  very  sensitive. 

Of  course,  there  is  a logical  explanation  for  this  finding.  Model  One  is 
almost  totally  insensitive  to  personalistic  data  as  discussed  in  Chapter  3 and 
this  fact  shows  up  in  the  performance  of  the  route  selection  model.  Thus, 
whether  one  were  studying  route  selection  with  personal  data  or  population  esti- 
mates should  make  little  difference.  One  might  note  also  that  on  the  average, 
the  route  selection  model  using  the  Acid  Test  Model  appears  to  be  more  sensi- 
tive to  population  variance  than  it  does  with  Model  Three.  This  tendency  was 
also  anticipated  in  Chapter  3. 
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Another  interesting  observation  from  Table  E.2  is  that  for  almost  every 
subject,  route  1 alternatives  appear  to  duplicate  themselves  most  often.  The 
most  reasonable  explanation  is  that  route  1 started  from  the  left  side  of  the 
battlefield  and  included  an  attack  upon  the  closest  target.  Thus,  there  might 
have  existed  fewer  reasonable  alternatives  for  route  1 than  for  the  other  routes. 

This  is  especially  true  for  subject  6 and  for  subjects  8 and  12  (see  the  tactical 
situation  layouts  in  Appendix  A). 

Analysis  of  Table  E.3  shows  that  the  degree  of  duplication  as  a function  of 
the  perception  model  used  is  much  less  pronounced,  although  some  duplication 
still  exists.  First,  Model  One  produced  results  that  were  unique  in  comparison 
to  either  of  the  other  two  models.  Thus,  Table  E.3  shows  only  the  duplication 
produced  by  Model  Three  and  the  Acid  Test  Model.  One  observation  to  be  made 
from  this  finding  is  that  there  is  some  indication  that  the  route  selection  model 
operates  with  Model  One  in  a fashion  that  is  distinct  from  its  operation  with 
either  of  the  other  two  models.  Such  a conclusion  would  not  be  too  surprising 
considering  the  definitions  of  the  models  in  Chapter  3. 

Also  from  Table  E.3,  there  is  some  indication  that  differences  between 
the  tactical  situations  for  which  the  route  selection  model  determines  routes  can 
influence  the  degree  of  similarity  in  results  produced  by  the  Acid  Test  Model 
and  Model  Three.  For  example,  in  the  results  produced  by  the  super  subject 
data  set,  there  is  no  variability  attributable  to  differences  in  subjects.  Therefore, 
a pure  situation  effect  can  be  determined.  Wc  note  then  that  the  subject  pair  (7, 11) 
produces  the  same  similarity  results,  the  subject  pair  (8,  12)  produces  the  same 
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similarity  results,  and  had  data  for  subjects  9 and  10  been  included,  similar 
findings  would  have  applied  to  the  pairs  (6,  10)  and  (9,  13).  Given  this  back- 
ground, it  would  appear  that  the  Acid  Test  Model  produces  results  that  differ  more 
often  from  those  produced  by  Model  Three  in  the  situations  faced  by  subject  pair 
(7,  11)  and  by  subject  pair  (9,  13).  Introducing  subject  variation  does  not  change 
this  observation.  In  fact,  with  subject  variation  the  amount  of  duplication  increases 
generally,  with  the  most  severe  increase  appearing  in  the  situation  faced  by  sub- 
ject pair  (8,  12).  No  adequate  explanation  can  be  offered  for  the  above  observations 
other  than  the  situations  faced  by  subject  pairs  (7,  11)  and  (9,  13)  may  be  more 
complex  and  hence  offer  a larger  number  of  reasonable  route  alternatives.  How- 
ever, the  most  important  conclusion  to  be  drawn  is  that  the  situation  for  which  a 
route  is  selected  may  influence  the  performance  of  the  model.  However,  the 
effect  may  not  be  as  great  as  that  produced  by  varying  the  input  data  set. 

Table  E.4  presents  a summary  of  similarity  generated  by  comparing 
alternatives  presented  to  each  subject  in  a subject  pair.  Of  course,  the  analysis 
applies  only  to  data  set  1 (personal  data  set)  since  data  set  2 (super  subject  data 
set)  produces  100%  similarity  in  all  cases.  Table  E.4  was  prepared  assuming 
that  for  each  subject-route  combination,  six  alternatives  were  actually  presented 
based  on  the  combinations  of  factor  levels  that  were  being  represented  (3  levels 
of  M and  2 levels  of  A).  If  in  fact  an  alternative  turned  out  to  be  missing  because 
the  factor  level  combination  did  not  produce  a unique  alternative,  then  the  alter- 
native thatwas  missing  was  of  course  treated  as  being  identical  to  the  one  that  was 
actually  presented. 
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Overall,  it  can  be  seen  that  a large  amount  of  duplication  exists,  with 
over  half  of  the  observations  being  duplicated.  Moreover  the  subject  pair  (7,  11) 


I 
I 

I I 

^ experienced  less  duplication  than  the  subject  pair  (8,  12);  and  route  1 produced 

more  duplication  than  the  other  routes,  with  route  2 a close  second.  These 
last  two  findings  are  in  agreement  with  those  reported  earlier. 

Now,  a large  amount  of  duplication  would  be  desired  if  one  were  wanting 
to  measure  a true  subject  effect  as  far  as  evaluation  of  the  route  selection  model 
is  concerned.  Since  almost  half  of  the  observations  are  not  duplicates,  analyses 
conducted  to  determine  the  effects  of  subject  to  subject  variation  would  be  largely 
in  error  if  the  analyses  were  based  upon  results  obtained  by  using  data  set  1. 

The  more  logical  choice  would  be  the  set  of  results  obtained  from  data  set  2 
because  100%  duplication  occurs. 

The  overall  observation  to  be  made  is  that  the  intrapersonal  variation 


measured  in  Phase  I does  not  appear  to  produce  extremely  large  variations  in 
the  performance  of  the  route  selection  model.  Thus,  this  observation  is  in  agree- 
ment with  that  reported  earlier.  Moreover,  if  the  effects  of  Model  One  were 

' 

separated  from  the  effects  of  the  other  two  models  as  was  done  earlier  it  is  felt 
that  most  of  the  similarity  could  be  attributed  to  Model  One. 

Analyses  of  Variance.  To  solve  the  problem  of  missing  data  so  that  analy- 
ses of  variance  could  be  conducted,  an  approach  similar  to  that  followed  during 
the  analysis  of  Phase  I data  was  taken.  That  is,  ior  each  data  cell  in  which  an 
observation  was  missing,  we  took  as  an  estimate  of  the  observation  the  value  that 
was  actually  realized  during  Phase  II.  For  example,  subject  G was  presented 
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only  five  alternatives  for  route  1.  The  data  which  were  realized  during  Phase 
EE  were  as  follows: 


Personal  Data 

Super  Subject  Data 

AT 

1 

3 

AT 

i 

3 

RF 

HF 

RF 

HF 

RF 

HF 

RF 

HF 

RF 

HF 

RF 

HF 

0 

0 



0 

0 

0 

0 

| 

D 

D 

B 

B 

0 

where  0 indicates  missing  data.  From  Table  E.  2,  however,  values  for  the 
missing  cells  could  be  determined  by  associating  factor  level  combinations  with 
alternatives.  Thus,  the  value  for  alternative  2 was  entered  not  only  in  the  cell 
associated  with  the  super  subject  data  set  (data  set  2)  but  also  in  the  cell  associ- 
ated with  the  personal  data  set  (data  set  1) , and  so  on.  The  same  procedure 
was  followed  for  each  subject-route  combination.  The  final  results  appear  in 
Table  E.5. 

Now,  no  estimate  of  the  actual  amount  of  error  introduced  by  the  missing 
data  procedure  is  available.  However,  the  possibility  for  error  is  indicated  by 
Table  E.  1.  Moreover,  one  may  infer  the  types  of  analyses  that  should  probably 
be  conducted  based  on  the  previous  discussion  of  Tables  E.  2 through  E.  4.  \lso, 
one  may  be  guided  in  his  interpretation  of  results  by  the  same  discussion. 

First  of  all,  a true  indication  of  differences  between  subjects  can  be 
measured  only  if  one  uses  the  data  produced  by  data  set  2 (super  subject)  since 
only  in  this  case  were  both  subjects  in  a subject  pair  presented  with  the  same 
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set  of  alternatives  to  evaluate.  Lack  of  significance  using  data  set  1 could  not 
be  considered  to  deny  the  existence  of  a subject  effect.  However,  a strong 
subject  effect  would  be  indicated  by  even  low-level  significance. 

Next,  it  is  difficult  to  measure  a data  set  effect  since  so  much  duplication 
occurs  between  sets  for  all  subjects  and  routes.  Interpretation  of  results  should 
be  guided  by  the  observations  that  were  made  with  respect  to  measurement  of  a 
subject  effect  with  data  set  1. 

Measurement  of  a true  perception  model  effect  can  be  accomplished  only 
through  an  analysis  of  data  from  subjects  7 and  13  (both  data  sets)  and  from  sub- 
ject 11  (data  set  2)  since  only  in  these  cases  do  the  Acid  Test  Model  and  Model 
Three  produce  no  duplications  (see  Table  E.3).  However,  degradation  of  the 
analysis  becomes  severe  only  for  subjects  8 and  12,  especially  in  the  case  of 
data  set  1, 

The  final  two  main  effects  (route  and  attack  method)  can  be  studied  in  any 
convenient  way  since  no  degradation  occurs  because  of  missing  data.  However, 
an  exact  analysis  of  any  two-factor  interaction  would  be  very  difficult  to  conduct, 
and  with  inexact  analyses  interpretation  of  such  results  would  be  even  more  dif- 
ficult. 

In  the  interest  of  brevity  we  present,  in  Figures  9 through  12,  a summary 
of  significant  results  obtained  through  various  analyses  of  variance.  Detailed 
analysis  of  variance  tables  corresponding  to  this  summary  appear  in  Appendix  E. 
Also,  plots  of  marginal  means  are  presented  in  Appendix  E to  indicate  trends 
corresponding  to  most  of  the  significant  results.  The  reader  should  bear  in  mind 
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Figure  9 . — Summary  of  Significant  Results  by  Subject 
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Data  Set  2 


* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0.01 


Figure  10. — Summary  of  Significant  Results  by  Subject  and  Data  Set 
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Figure  ll. — Summary  of  Significant  Results  by  Subject  Pair 
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Figure  12. — Summary  of  Significant  Results  by 
Subject  Pair  and  Data  Set 


that  the  analyses  are  only  approximate  because  of  the  steps  taken  to  account 


for  missing  data. 

In  the  paragraphs  that  follow  we  discuss  conclusions  to  be  drawn  from 
Figures  9 through  12  and  the  associated  plots  in  Appendix  E.  However,  prior  to 
this  discussion,  let  us  make  some  further  observations  concerning  the  interpre- 
tation of  significance.  In  Figures  13  and  14  are  presented  plots  of  a performance 
measure  by  subject.  This  performance  measure  P is  the  ratio  of  sums  of  squares 
of  error  to  total  sums  of  squares  and  may  be  viewed  as  a measure  of  the  degree 
to  which  the  statistical  model  fits  the  data.  It  can  be  seen  that  relatively  large 
amounts  of  unexplained  error  exist  for  subjects  11  and  13  considering  data  set 
1 alone,  while  the  same  is  true  for  subjects  7 and  13  considering  only  data  set 
2.  Overall,  subjects  11  and  13  exhibited  the  greatest  amount  of  variability  with 
respect  to  the  model,  with  subjects  6 and  8 performing  best.  In  any  case,  with 
little  unexplained  variability  it  takes  less  factor  variability  to  produce  a sig- 
nificant result.  Thus,  for  example,  one  would  expect  many  significant  results 
from  subject  8,  especially  with  data  set  1.  The  summary  tables  do  reveal  this  trend. 
Data  Set  Effect 

As  may  be  seen  from  Figure  9 the  data  set  effect  is  significant  for  half 
the  subjects,  and  furthermore  it  interacts  strongly  with  perception  model  effect 
for  most  subjects.  Examination  of  the  marginal  means  in  Appendix  E indicates 
that  for  all  subjects  except  subject  13,  alternatives  selected  by  using  personal 
data  are  preferred  over  those  selected  by  using  super  subject  data.  This  difference 
can  be  as  little  as  5%  or  as  great  as  15%.  However,  subject  13  exhibits  exactly 
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Figure  13.  — Plot  of  Performance  Measure  by  Subject 
(Combined  Data  Sets) 


Figure  14.  — Plot  of  Performance  Measure  by  Subject 
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the  opposite  trend  with  over  15%  greater  preference  for  data  set  2.  Considering 

• 

the  interactions  that  exist  in  addition  to  the  main  effect,  one  would  be  led  to  con- 
clude that  most  pilots  prefer  results  produced  by  their  own  threat  perception  data 
as  opposed  to  those  produced  by  population  estimates.  In  any  case,  population 
estimates  are  not  suitable  as  input  data,  especially  if  one  were  to  use  Model 
Three  or  the  Acid  Test  Model  (see  plot  of  interactions).  Thus,  it  appears  that 
one  should  always  use  input  data  that  apply  to  an  individual  decision  maker  if  one 
wishes  to  increase  his  chances  of  obtaining  results  from  the  route  selection 
model  that  are  more  acceptable  to  the  decision  maker  involved. 

Route  Effect 

One  of  the  strongest  effects  in  the  experiment  is  the  route  effect.  In  the 
analysis  that  includes  the  data  set  factor,  the  effect  is  significant  for  all  subjects. 
Furthermore,  it  is  significant  for  at  least  half  the  subjects  even  when  indepen- 
dent analyses  are  performed  at  each  level  cf  the  data  set  factor.  Moreover,  a 
considerable  amount  of  interaction  occurs  between  this  effect  and  other  effects, 
especially  the  perception  model  effect.  Thus,  one  is  led  to  conclude  that  the 
subjects  view  the  results  of  the  model  differently  as  the  route  planning  task 
changes.  Hence,  the  performance  of  the  model  is  a function  of  the  situation  in 
which  it  is  to  be  applied.  This  is  a very  unfortunate  conclusion,  for  one  would 
desire  the  model  to  perform  equally  well  under  all  conditions. 

Now,  for  most  subjects  the  model  appears  to  work  best  for  routes  3 and 
4 as  opposed  to  routes  1 and  2.  As  explained  earlier,  the  difference  between 
these  routes  was  that  routes  1 and  2 include  an  attack  upon  the  target  closest  to 


142 


the  left  hand  side  of  the  battlefield.  However,  they  also  differed  in  that  the 
target  to  be  attacked  during  execution  of  these  routes  was  usually  the  lighter  of 
the  two  weapons  present.  Therefore,  it  would  appear  that  the  model  performs 
better  with  a more  lethal  weapon  as  the  target,  or  perhaps  the  model  performs 
best  when  the  subject  percieves  the  route  selection  situation  to  be  more  difficult. 
Such  a tendency  is  indicated  by  the  data  in  Table  5.  These  data  were  collected 
during  the  first  step  of  Phase  II  in  which  the  subjects  were  asked  to  rank  and 
rate  the  four  route  selection  situations  on  the  basis  of  difficulty,  with  the  most 
difficult  route  rated  100.  One  can  see  that  the  data  correspond  generally  with 
the  ratings  given  the  route  alternatives.  However,  more  investigations  are 
required  in  this  area . 


Table  5 

Relative  Route-Situation  Ratings 


Subject 

Route-Situation 

1 

2 

3 

4 

6 

45 

89 

70 

100 

7 

90 

50 

75 

100 

8 

20 

85 

45 

100 

9 

24 

47 

88 

100 

10 

22 

33 

89 

100 

11 

62 

100 

94 

94 

12 

25 

62 

87 

100 

13 

60 

100 

40 

85 

Normalized 

Average 

45 

73 

76 

100 
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Attack  rode 

The  attack  mode  effect  seems  to  be  fairly  significant  with  only  a 
relatively  small  number  of  interactions  occurring  with  the  other  factors . It  is 
interesting  to  note  that  for  the  subjects  registering  a significant  effect,  running 
fire  route  alternatives  are  preferred  to  hover  fire  alternatives.  Thus,  it  may 
be  that  the  method  used  in  the  route  selection  model  to  represent  running  fire  is 
better  than  that  used  to  represent  hover  fire.  More  on  this  topic  will  be  pre- 
sented in  Chapter  5. 

Perception  Model 

The  perception  model  effect  is  also  one  of  the  most  significant 
effects  in  the  experiment  as  we  had  hoped  it  would  be . Unfortunately  interactions 
exist  with  the  route  effect  and  also  with  the  subject  effect. 

Examination  of  the  marginal  mean  plots  indicates  there  is  a 

large  overall  difference  between  Model  One  and  the  other  two  perception  models 
taken  together.  From  the  preceding  analysis  of  missing  data,  it  is  understand- 
able why  the  Acid  Test  Model  and  Model  Three  score  so  closely  together  (see 
Table  E.3)  in  relation  to  Model  One.  However,  the  magnitude  of  the  difference 
exhibited  between  Model  One  and  the  other  two  models  is  pleasantly  surprising. 
One  can  only  conclude  that  Model  One  results  are  evaluated  differently  by  sub- 
jects , and  from  the  main  effects  plot,Model  One  results  are  preferred  by  four 
of  the  six  subjects.  Subject  13  prefers  Model  Three  results  while  subject  12 
prefers  either  of  the  other  two  model  results. 
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Now,  the  interaction  with  the  route  effect  causes  some  problem,  but  it 
is  not  overwhelming.  Examination  reveals  that  the  model  that  is  preferred  by 
a subject  will  in  most  cases  continue  to  be  preferred  as  the  routes  change , even 


I 
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To  test  the  effects  of  the  interaction,  a minimax  analysis  was  performed. 
First,  the  route -perception  model  marginal  means  were  averaged  across  all  sub- 
jects. The  resulting  data  appear  as  in  Figure  15.  A minimax  solution  suggests  that 
Model  One  should  be  adopted  in  the  face  of  uncertainity  with  respect  to  the  situation 
in  which  the  model  would  be  applied.  If  Model  One  were  adopted  the  worst  result 
would  be  an  average  rating  of  56%.  However,  to  choose  the  Acid  Test  Model  could 
yield  an  average  rating  of  39%  while  a choice  of  Model  Three  could  yield  an  aver- 
age rating  as  low  as  44%. 


A word  of  caution  should  be  interjected  at  this  point.  To  select  Model 
One  to  the  exclusion  of  the  other  models  could  be  a foolish  decision  in  the  face 
of  the  rather  sizable  perception  model-subject  interaction.  From  the  marginal 
mean  plots  it  is  known  that  two  of  the  six  subjects  preferred  a model  other  than 
Model  One.  In  fact,  we  can  be  only  65%  confident  that  we  would  please  even  half 
the  entire  Army  pilot  population  possessing  the  backgrounds  of  the  subjects  tested. 
With  different  types  of  training  and  experience  the  subjects  might  view  the  models 
in  an  entirely  different  fashion.  A more  prudent  approach  would  be  to  select 
Model  One  as  the  most  promising  candidate  but  to  continue  experimentation  to 
determine  if  alterations  of  the  models  tested  so  far  can  produce  better  results . 

At  the  very  least,  another  subject  group  with  a possibly  different  background 
should  be  tested  in  an  attempt  to  improve  the  confidence  limit.  More  on  this  topic 
will  be  presented  in  Chapters  5 and  6. 

Subject  Effect 

In  all  the  preceding  analyses,  numerous  occasions  have  arisen  in  which 
the  subject  factor  interacted  with  another  factor.  Examination  reveals  that  in 
addition  to  these  interactions  a strong  subject  effect  occurred  in  most  of  the 
analyses  of  variance,  although  the  main  effect  was  not  as  important  as  the  inter- 
actions in  general.  Figures  16  through  18  support  this  contention.  Here,  we 
have  presented  a breakdown  of  the  various  analyses  of  variance  that  were  per- 
formed in  which  the  subject  factor  was  present.  The  intent  is  to  show  the  con- 
tribution of  subject  main  effect  and  subject  interactions  to  overall  variation.  In 
particular  it  is  seen  that  subjects  8 and  12  exhibited  basically  two  different 
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Subjects  7 and  11 


^Percent  of 
Total 


Source  of  Variation 


Main  Effects 
Subjects 
Other 


Interactions 

Subject 

Other 

Residual 

V 


Degrees  of  Fx-eedom 
8 

1 

7 

24 

7 

17 


Sums  of  Squares 


35138.37 

4280.01 

30858.36 


41.0 


5.0 

36.0 


25972.06 

2332.30 

23639.76 


30.4 


2.7 

27.7 


24524.69 


28.6  28.6 


TOTAL  95 


85635.12  100.0  100.0 


Subjects  8 and  12 

Percent  of 

Source  of  Variation 

Degrees  of  Freedom 

Sums  of  Squares 

Total 

Main  Effects 

8 

18070.95 

17.8 

Subject 

1 

8990.01 

8.S 

Other 

1 

9080.94 

8.S 

Interactions 

24 

66735.84 

65.8 

Subject 

7 

47372.38 

46.6 

Other 

17 

19363.96 

19.2 

Residual 

63 

16624.96 

16.4  16.4 

TOTAL 

95 

101431.75 

100.0  100.0 

Figure  16. — Subject  Effects  and  Interactions 
(Combined  Data  Sets) 
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Subjects  7 and  11  (Data  Set  1) 

Percent  of 

Source  of  Variation  Degrees  of  Freedom  Sums  of  Squares  Total 


Main  Effects 

7 

13499.07 

40.3 

Subject 

1 

4485.33 

13.4 

Other 

6 

9013.74 

26.9 

Interactions 

17 

11640.43 

34.7 

Subject 

6 

3132.76 

9.3 

Other 

11 

8507.67 

25.4 

Residual 

23 

8378.36 

25.0 

25.0 

TOTAL 

47 

33517.85 

Subjects  7 and  11  (Data  Set  2) 

Percent  of 

Source  of  Variation 

Degrees  of  Freesom 

Sums  of  Squares 

Total 

Main  Effects 

7 

23768.13 

47.1 

Subject 

1 

652.69 

1.3 

Other 

6 

23115.44 

45.8 

Interactions 

17 

15696.52 

31.2 

Subject 

6 

1875.07 

3.7 

Other 

11 

13821.45 

27.5 

Residual 

23 

10928.27 

21.7 

21.7 

TOTAL 

47 

50392.89 

100.0 

100.0 

Figure  17 . — Subject  Effects  and  Interactions 
by  Data  Set  (Subjects  7 and  11) 
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Subjects  8 and  12  (Data  Set  1) 


Percent  of 

Source  of  Variation  Degrees  of  Freedom  Sums  of  Squares  Total 


Main  Effects 

7 

13647.61 

25.1 

Subject 

1 

4275.19 

7.9 

Other 

6 

9372.42 

17.2 

Interactions 

17 

37346.77 

68.7 

Subject 

6 

28312.87 

52.1 

Other 

11 

9033.90 

16.6 

Residual 

23 

3353.55 

6.2 

6.2 

TOTAL 

47 

54347.90 

100.0 

100.0 

Subjects  8 and  12  (Data  Set  2) 

Percent  of 

Source  of  Variation 

Degrees  cf  Freedom 

Sums  of  Squares 

Total 

Main  Effects 

7 

8561.12 

18.5 

Subject 

1 

4720.33 

10.2 

Other 

6 

3840.79 

8.3 

Interactions 

17 

29243.98 

63.4 

Subject 

6 

21318.06 

46.2 

Other 

11 

7925.92 

17.2 

Residual 

23 

8348.77 

18.1 

18.1 

TOTAL 

47 

46153.84 

100.0 

100.0 

Figure  18. — Subject  Effects  and  Interactions 
by  Data  Set  (Subjects  8 and  12) 
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modes  of  decision  behavior.  Further  examination  of  many  of  the  marginal  mean 


plots  shows  that  subject  12  acts  in  a manner  very  similar  to  subject  13,  while 
subject  8 performs  as  subjects  6,7,  and  11.  Thus,  we  are  led  to  conclude  that 
at  least  two  and  possibly  more  basic  modes  of  decision  behavior  may  have  existed 
within  the  subject  group.  If  this  were  found  to  actually  be  the  case  in  general,  then 
any  hope  of  developing  one  descriptive  route  selection  model  that  would  be  univer- 
sally applicable  to  all  helicopter  pilots  would  be  dashed.  Also,  such  a finding 
would  possibly  possess  widespread  implications.  For  example,  the  philosophy 
of  combat  operations  simulation  design  would  be  changed  drastically.  Further- 
more, if  some  of  the  modes  of  behavior  could  be  demonstrated  to  be  in  some 
way  undesirable,  then  Army  training  programs  or  pilot  screening  procedures 
might  need  to  be  revised.  Unfortunately  the  limited  experimentation  that  has  been 
conducted  does  not  permit  strong  statistical  statements  in  this  area. 

Worth  of  the  Model 

As  has  been  seen,  the  rating  given  to  the  route  selection  model  by  the 
subject  group  varies  in  a very  complex  way  with  almost  every  factor  included  in 
the  analysis.  However,  some  very  gross  observations  may  be  made  with  res- 
in | to  the  int  i ill  worth  of  the  model  as  viewed  by  the  decision  makers.  Figure 
19  shows  the  rating  of  the  route  selection  model  as  a function  of  the  perception 
model  used,  for  two  different  classes  of  decision  makers  (those  preferring  Model 
One,  those  not  preferring  Model  One)as  well  as  for  all  decision  makers.  The 
effects  of  all  other  factors  have  been  averaged.  Recalling  that  the  ratings  were: 
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Perception  Model 


Figure  19. — Overall  Model  Rating 


100:  equivalent  to  decision  maker's  route  plan, 

95-99:  decision  maker's  plan  preferred,  but  no 
reason  can  be  given  for  preference, 

70-94:  decision  maker's  plan  slightly  preferred, 
with  reason,  and 

0-69:  decision  maker's  plan  strongly  preferred 
with  reason. 

Thus,  among  subjects  preferring  Model  One,  the  decision  maker's  plan  was  only 
slightly  more  preferable  than  the  route  selection  model  alternatives.  However, 
this  same  model  would  be  rejected  strongly  by  those  subjects  preferring  one  of 
the  other  models.  The  same  observations  can  be  made  about  the  Acid  Test- 
Model  Three  combination.  However,  if  one  were  to  try  to  please  everyone  with 
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just  one  model,  then  on  the  average  exactly  the  opposite  effect  would  be  achieved. 
This  observation  is  in  agreement  with  observations  made  earlier  and  more  on  this 
topic  will  be  presented  in  the  next  chapter. 


T 


Effect  of  Approximations 

In  Chapter  3 it  was  pointed  out  that  the  route  selection  model  as  it  exists 
can  only  approximately  select  routes  according  to  the  Acid  Test  Model  of  per- 
ceived threat.  This  is  so  because  a decision  maker  who  actually  perceives 
threat  according  to  the  Acid  Test  Model  does  not  perceive  route  segment  threats 
in  an  additive  manner.  That  is,  segment  disutilities  are  not  additive.  To  revise 
the  route  selection  model  to  account  for  this  deficiency  is  beyond  the  scope  of  the 
present  research.  However,  the  inadequacy  of  the  route  selection  model  may  in 
part  account  for  the  poor  showing  of  the  Acid  Test  Model.  Unfortunately,  at  this 
point  there  is  no  way  to  determine  to  what  degree  the  Acid  Test  Model  results 
have  been  degraded. 

It  is  also  true  that  the  route  selection  model  can  only  approximately 
select  routes  according  to  Model  Three.  This  is  so  if  a route  segment  intersects 
the  circle  defining  the  maximum  perceived  effective  range  of  an  enemy  weapon. 
For  example,  in  Figure  20,  a route  segment  of  duration  t has  its  midpoint 
inside  the  circle.  Assuming  only  the  one  weapon  in  the  scenario,  perceived 
threat  is  simply  t.  However,  if  the  segment  were  divided  at  its  midpoint  into 
two  subsegments  as  shown  then  the  perceived  threat  would  be  .001  ^ + tg^  .5005t 
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Thus,  the  perception  of  threat  predicted  for  route  segments  by  Model  Three  is 
dependent  upon  the  layout  of  the  convex  areas  whose  boundaries  form  route 
segment  end  points.  Instead  of  the  rather  arbitrary  approach  that  was  taken  in 
structuring  the  route  selection  environment  (see  Appendix  A),  care  should  have 
been  exercised  to  approximate  the  maximum  threatening  range  boundary  with 
edges  of  convex  areas.  Then  route  segments  would  terminate  on  the  threatening 
boundary  instead  of  intersect  it.  Unfortunately,  to  take  such  care,  tailoring  the 
representation  to  each  subject  would  have  been  prohibitively  time  consuming 
and  costly. 
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CHAPTER  5 

RESEARCH  EXTENSIONS  TO  THE 
FORT  RUCKER  EXPERIMENT 

By  Don  C . Hutcherson 

Introduction 

The  reader  will  recall  from  Chapter  1 that  one  of  the  purposes  of  this 
research  is  to  determine  whether  or  not  decision  and  value-theory  concepts  can 
be  used  as  elements  of  a model  to  explain  the  behavior  of  a military  decision- 
maker in  a helicopter  route  planning  task.  Considering  the  results  presented 
in  Chapter  4 it  is  not  apparent  that  the  question  has  been  completely  resolved. 

First,  it  was  discovered  during  the  Phase  I Experiment  (Chapter  3)  that 
considerable  variation  exists  between  subjects,  not  only  with  respect  to  such 
topics  as  the  speed  that  is  to  be  employed  during  an  attack,  but  also  with  res- 
pect to  perceptual  topics  such  as  the  relative  threat  ratings  that  are  given  to 
a set  of  enemy  weapons.  Even  though  all  the  subjects  were  presented  the  same 
information  during  the  experiment  they  either  perceived  the  information  dif- 
ferently or  they  were  operating  on  different  sets  of  preconceived  notions  developed 
individually  or  perhaps  in  groups. 

Now,  such  a finding  is  not  bad  in  itself  since  the  hypothesis  had  been  made 
at  the  start  of  the  research  program  that  the  population  of  pilots  might  very  well 
be  heterogeneous;  the  experiments  had  been  designed  to  test  this  hypothesis. 
However,  during  the  Phase  II  Experiment  it  was  diseovered  that  heterogeneity 


apparently  carries  over  even  into  the  methods  used  by  pilots  to  select  routes. 
For  example,  a highly  significant  subject-perception  model  interaction  was 
found  to  exist.  Moreover,  four  of  the  six  subjects  apparently  preferred  routes 
determined  by  the  route  selection  model  employing  Model  One  to  measure  per- 
ceived threat.  These  subjects  as  a group  found  their  own  routes  to  be  only 
slightly  more  preferable  overall  than  the  routes  selected  by  the  computer  using 
Model  One.  However,  the  same  model  was  rejected  strongly  by  the  other  two 
subjects.  These  subjects  much  preferred  routes  selected  by  using  either  Model 
Three  or  the  Acid  Test  Model. 

The  net  result  of  the  above  heterogeneity  is  that  it  may  be  impossible  to 
develop  a single  route  selection  model  that  is  completely  acceptable  to  all  pilots. 
Even  if  the  model  were  designed  to  accept  personalistic  input  data  from 
individual  pilots  it  still  might  not  produce  results  that  are  acceptable  to  all 
individuals.  Thus,  two  or  more  route  selection  models  may  be  required. 

Another  result  of  the  Phase  II  Experiment  is  that  the  acceptability  of  the 
computer  prepared  routes  was  not  as  high  as  had  been  hoped  for  generally. 
Overall,  the  six  subjects  rated  the  computer  routes  as  being  less  that  70%  as 
good  as  their  own,  which  translates  into  the  following  qualitative  statement 
(using  the  rules  of  Chapter  4):  the  decision-maker  strongly  prefers  his  routes 
to  those  prepared  by  the  model  and  can  state  definite  reasons  for  his  preference. 
Even  if  the  average  rating  for  Model  One  is  determined  by  using  only  the  ratings 
from  the  four  subjects  who  apparently  preferred  Model  One  results,  a value  of 
less  Ilian  !)0‘,  is  obtained.  This  translates  into  the  statement  that  the  decision- 
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maker  only  slightly  prefers  his  own  route  plan,  but  again  a reason  can  be  cited 
for  the  preference. 


All  of  the  observations  cited  above  lead  to  the  conclusion  that  further 
analysis  and  experimentation  are  required  to  obtain  a better  understanding  of 
the  route  selection  decision  process.  The  remainder  of  this  chapter  is  devoted 
to  a discussion  of  a first  step  that  has  been  taken  in  this  direction. 

First,  analyses  will  be  described  that  were  conducted  in  an  attempt  to 
isolate  those  factors  that  were  instrumental  in  the  subjects'  rejection  of  the 
computer  prepared  routes  in  favor  of  their  own  routes.  Then,  modifications 
that  were  made  to  the  route  selection  model  in  an  attempt  to  obtain  routes  of 
higher  acceptability  will  be  described.  Next,  an  experiment  will  be  discussed 
which  had  two  objectives.  First,  the  experiment  was  conducted  to  determine 
whether  or  not  the  modifications  to  the  route  selection  model  had  had  any  effect 
on  the  acceptability  of  the  model's  routes.  Second,  the  experiment  was  con- 
ducted to  determine  if  a second  indication  of  population  heterogeneity  could  be 
obtained,  and  if  so,  whether  the  second  indication  was  consistent  with  the  first. 
Finally,  an  analysis  of  the  results  of  the  experiment  will  be  presented  along 
with  conclusions  that  were  drawn  from  the  analysis. 

Analysis  of  Rejection  Factors 

The  fact  that  several  of  the  subjects  participating  in  the  Phase  I and 
Phase  II  Experiments  consistently  displayed  a high  degree  of  enthusiasm  for 
their  tasks  turned  out  to  be  extremely  important  to  the  analysis  of  factors  lead- 
ing to  rejection  of  some  of  the  route  selection  model  results.  First,  these 
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subjects  submitted  copious  notes  to  explain  their  reasons  for  rejecting  com- 
puter routes  they  judged  to  violate  one  or  more  of  their  route  selection  criteria. 


An  example  of  some  of  these  notes  appears  in  Table  6. 

Next,  the  subjects  were  very  conscientious  in  completing  a written 
critique  of  the  experiment  that  was  administered  at  the  end  of  the  first  experi- 
mental session,  immediately  after  the  route  selection  exercise.  While  the 
critique  covered  a variety  of  topics,  responses  to  questions  related  to  the 
route  selection  decision  process  turned  ~ut  to  be  especially  helpful  in  the 
analysis  of  rejection  factors.  A detailed  summary  of  responses  to  all  questions 
in  the  crituque  appears  in  Appendix  F at  Exhibit  F.  1. 

Finally,  additional  insight  was  obtained  from  responses  to  one  of  the 
questions  contained  on  the  Master  Information  Form  which  was  completed  during 
the  Phase  I Experiment.  This  question  was  concerned  with  the  factors  that  each 
subject  considered  in  deciding  the  mode  of  fire  (running  fire  or  hover  fire)  he 
would  employ  during  a given  attack.  A summary  of  responses  to  this  question 
appears  in  Table  7. 

In  analyzing  material  that  relates  to  the  overall  route  selection  decision 
process  in  Table  6 and  Exhibit  F.  1,  it  is  difficult  to  isolate  anything  in  particular 
that  would  cause  the  subjects  generally  to  reject  route  selection  model  results. 

In  fact,  the  subjects  apparently  agree  with  many  of  the  assumptions  that  were 
made  in  developing  the  route  selection  model  in  general  and  with  the  assumptions 
of  Model  One  as  a threat  perception  model  in  particular.  In  summary,  the 
subjects  stale  that  they  choose  their  routes  to  minimize  exposure  to  enemy 
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Table  6 


f 


Summary  of  Notes  on  Rejection  Criteria 


This  route  approaches  within  the  effective  range  of  the  supporting  weapon. 

There  is  no  need  to  engage  in  a running  fire  duel  when  a covered  route  to  a 
hover  fire  position  is  available. 

I would  never  select  a hover  fire  position  that  requires  an  exposed  approach 
to  the  position. 

This  route  docs  not  use  the  terrain  as  well  as  my  route. 

In  order  to  fire  from  the  indicated  firing  position  one  would  have  to  climb  too 
high.  This  means  the  aircraft  would  be  exposed  too  long  and  would  be 
silhouetted  when  the  firing  position  is  achieved. 

The  hover  fire  position  is  too  close  to  the  target. 

Running  fire  in  this  situation  results  in  too  much  exposure. 

This  route  takes  a short  cut  that  is  exposed. 
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Table  7 


Summary  of  Responses  Regarding  Factors 
Leading  to  a Choice  in  Attack  Mode 


Attack  Mode 

Factors  Cited 

Hover  Fire 

Masking  available  during  approach  to  firing  posi- 
tion. 

Minimum  exposure  to  target  weapon. 

Minimum  exposure  to  supporting  weapon. 

Avoids  being  silhouetted  against  the  sky. 

Outside  target's  effective  range. 

Outside  supporting  weapon's  effective  range. 

Permits  engagement  at  maximum  range  of  my 
weapon. 

Targets  are  hard  to  detect  during  running  fire. 
Permits  fire  as  soon  as  range  is  "right." 

Running  Fire 

Attack  must  be  conducted  inside  target's  effective 
range. 

No  masking  terrain  available . 

The  enemy  cannot  hit  a moving  target  as  well  as  a 
stationary  target. 

Launch  range  is  too  close  for  hover  fire. 

Minimum  exposure  time . 

Avoids  being  silhouetted  against  the  sky. 

Allows  movement  to  masked  area  while  missile  is 
still  in  flight. 
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weapons  and  maximize  the  element  of  surprise  during  an  attack.  During  enroute 
travel  they  fly  as  close  to  the  ground  as  possible  and  will  go  to  any  length  to 
avoid  being  intervisible  with  the  enemy.  If  intervisibility  is  unavoidable  they 
try  to  choose  a path  that  is  as  short  as  possible  and  as  far  as  possible  from  the 
enemy  weapon.  For  some,  even  an  exposure  time  that  is  less  than  the  reaction 
time  of  the  enemy  weapon  is  considered  dangerous. 

Proceeding  further  in  the  analysis  of  material  in  Tables  6 and  7 and 
Exhibit  F.l,  one  finds  that  the  subjects  apparently  have  very  definite  and  fairly 
consistent  criteria  for  selecting  a mode  of  fire  (running  or  hover)  to  employ  in 
a given  tactical  situation.  The  same  can  be  said  for  the  criteria  they  use  in 
selecting  a firing  position.  For  example,  the  subjects  apparently  favor  hover 
fire  over  running  fire  unless  they  are  forced  into  a duel  by  some  factor  such  as 
terrain  (lack  of  covered  approaches)  or  weapon  effective  range  (their  own  versus 
the  enemy's).  They  apparently  would  prefer  to  deliver  fire  from  a hover  fire 
position  outside  the  range  of  the  enemy  weapon  if  possible.  Furthermore,  in 
choosing  a hover  fire  position  they  seek  a masked  avenue  of  approach  from  which 
they  may  "pop  up"  to  fire.  However,  they  try  to  avoid  being  silhouetted  against 
the  sky  from  this  position. 

Now,  it  is  in  this  area  that  there  may  be  some  problems  with  the  route 
selection  model.  It  is  obvious  that  the  subjects  considered  the  selection  of  a 
mode  of  fire  and  a firing  position  to  be  one  of  the  most  important  aspects  of  the 
route  selection  task.  Yet,  it  should  be  obvious  that,  to  this  point  in  time,  the 
research  program  lias  not  been  directed  specifically  toward  an  analysis  of  the 


decision  process  involved  in  the  selection  of  a mode  of  fire.  Furthermore,  as 
explained  in  a subsequent  paragraph,  the  procedures  used  in  the  model  to 
represent  the  selection  of  a firing  position  are  fairly  simple,  and  it  is  possible 
that  they  may  be  at  odds  with  the  actual  decision  process. 

In  the  remaining  paragraphs  of  this  section,  attention  wfll  be  directed 
first  to  a review  of  the  procedures  used  in  the  route  selection  model  to  repre- 
sent the  selection  of  a firing  position.  Then,  a discussion  of  modifications  to 
the  model  that  have  been  made  to  better  represent  the  fire  position  selection 
process  will  be  presented.  Next,  the  mode  of  fire  decision  process  will  be 
discussed  in  detail,  and  a procedure,  in  the  form  of  a hypothesis,  will  be  pro- 
posed to  predict  the  mode  of  fire  that  a subject  will  select  in  a tactical  situation 
of  the  type  that  was  analyzed  during  the  Phase  II  Experiment.  Finally,  an 
analysis  will  be  performed  to  determine  whether  there  is  any  relationship  between 
the  mode  of  fire  that  a subject  selects  and  the  way  he  views  the  route  selection 
model  results. 

Fire  Position  Procedures.  The  reader  will  recall  from  Chapter  2 that  routes 
constructed  by  the  route  selection  model  consist  of  five  phases.  The  first  is  an 
enroute  or  approach  phase  conducted  at  tree  top  level  (knap  of  the  earth)  from  a 
specified  starting  position  to  the  vicinity  of  the  target.  This  phase  consists  of 
one  or  more  straight  line  segments  connected  to  form  a continuous  route.  The 
next  three  phases  are  associated  with  an  attack  upon  the  target,  with  each  phase 
consisting  of  a single  straight  line  segment.  The  first  segment  represents  a 
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transition  from  the  approach  route  into  a firing  position  and  is  followed  by 


another  segment  to  represent  the  firing  activity.  The  final  attack  segment 
represents  a transition  into  the  escape  phase  of  the  mission.  The  escape  phase 
is  similar  to  the  approach  phase  in  that  it  is  conducted  at  tree  top  level  and 
consists  of  one  or  more  connected  straight  line  route  segments.  The  route 
terminates  at  a specified  boundary  of  the  battlefield. 

Now,  the  segment  chosen  to  represent  the  firing  activity  is  actually  only 
one  of  a number  of  segments  that  could  have  been  chosen  for  the  route,  hi 
preparing  each  route  selection  problem  for  execution  a series  of  equally  spaced 
rays  extending  outward  from  the  target  is  constructed  to  define  the  directions 
from  which  attacks  could  be  conducted.  Then  a series  of  points  is  placed  at 
equally  spaced  intervals  along  each  attack  ray  to  represent  the  points  at  which 
the  firing  activity  might  commence.  Finally,  there  is  associated  with  each  of 
the  firing  position  points  another  point  at  which  the  attack  might  terminate  (upon 
impact  of  a missile  that  has  been  launched).  Any  given  pair  of  points  forms  the 
endpoints  of  a possible  firing  activity  segment.  The  route  selection  model 
simply  determines  a route  that  contains  one  and  only  one  of  the  possible  firing 
activity  segments.  The  segment  chosen  is  the  one  that  permits  a route  of 
minimum  perceived  threat  to  be  constructed.  (See  Chapter  2.) 

Now,  a basic  requirement  of  the  model  is  that  one  must  be  able  to 
determine  the  perceived  threat  associated  with  any  given  route  segment. 
Furthermore,  regardless  of  the  perception  model  used,  the  perceived  threat 
i - - - luted  with  a route  segment  is  calculated  in  different  ways  depending  upon 
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whether  or  not  the  segment  is  intervisible  with  an  enemy  weapon.  (See  Chapter 


3.)  Thus,  one  must  always  be  able  to  specify  the  intervisibility  condition  that 
exists  on  a segment. 

For  the  enroute  phases  of  the  mission  this  requirement  poses  no  problem 
since  these  phases  are  conducted  at  tree  top  level,  and  intervisibility  conditions 
for  this  mode  of  flight  are  specified  at  all  points  on  the  battlefield  by  a line  of 
sight  map.  (See  Appendix  A and  Chapter  2.)  However,  there  is  a problem  in 
specifying  intervisibility  relationships  for  the  three  segments  comprising  an 
attack.  First,  it  is  obvious  that  the  firing  activity  segment  is  intervisible  with 
the  target  since  the  existence  of  a line  of  sight  has  been  specified  as  a condition 
for  an  attack.  However,  without  calculation,  it  is  impossible  to  determine 
whether  this  segment  is  intervisible  with  other  weapons  on  the  battlefield. 

Next,  the  transition  segments  may  involve  vertical  movement  in  relation 
to  the  terrain,  either  a climb  to  a firing  position  or  a dive  after  the  firing  activity. 
Thus,  a highly  dynamic  situation  exists  on  these  segments  with  respect  to  inter- 
visibility. Again,  nothing  can  be  determined  about  the  actual  conditions  that 
exist  without  calculation. 

To  simplify  procedures  in  the  route  selection  model  it  is  assumed,  for 
computational  purposes  only,  that  the  transition  segments  of  an  attack  are 
conducted  at  tree  top  level.  Thus,  the  intervisibility  conditions  that  exist  on 
these  segments  can  be  determined  by  the  same  procedures  that  arc  used  to 
determine  conditions  along  any  of  the  enroute  segments  of  the  mission.  Further- 
more, it  is  assumed  that  the  firing  activity  segment  is  fully  exposed  to  all  enemy 
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weapons  on  the  battlefield.  These  two  assumptions  permit  very  rapid  evaluation 
of  the  perceived  threat  associated  with  attack  segments.  Moreover,  at  the  time 
the  assumptions  were  made  it  was  felt  that  they  would  be  fairly  representative 
of  the  way  in  which  the  pilots  actually  attempt  to  conduct  an  attack  and  the  way  in 
which  they  view  the  inter  visibility  situation.  As  will  be  discussed  below,  there 
evidently  is  reason  to  doubt  these  assumptions. 

Model  Modifications.  In  examining  the  material  in  Tables  6 and  7 and  Exhibit 
F.l,  there  are  several  comments  from  the  subjects  which  lead  to  the  following 
general  conclusions.  First,  if  hover  lire  is  lx*ing  contemplated  the  firing  posi- 
tion is  evidently  selected  in  such  a way  that  the  route  leading  to  the  position  is 
completely  covered  with  respect  to  all  enemy  weapons . Otherwise,  hover  fire 
will  not  be  used.  Next,  hover  fire  is  apparently  conducted  from  a position  that 
is  at  or  near  the  maximum  effective  range  of  the  ordnance  being  used;  and 
hopefully  this  position  is  outside  the  range  of  the  target's  weapon.  Finally, 
some  of  the  subjects  evidently  recognize  that  in  climbing  to  a firing  position, 
there  is  a good  possibility  that  the  transition  segment  may  be  exposed  to  other 
weapons  on  the  battlefield.  Furthermore , the  exposure  time  involved  may  be 
quite  large  since  some  firing  positions  must  be  highly  elevated  above  the  terrain 
to  achieve  a line  of  sight  with  the  target. 

From  the  previous  discussion  of  the  firing  position  selection  procedures, 
it  is  obvious  that  there  are  dcscrepancies  between  the  procedures  used  in  the 
route  selection  model  and  the  methods  used  by  the  subjects.  Therefore,  the 


1 65 


model  procedures  have  been  modified  in  an  attempt  to  represent  the  decision 


I First,  routes  involving  hover  fire  are  constructed  to  insure  that  the 

approach  to  a firing  position  is  as  covered  as  possible.  To  accomplish  this 
objective,  the  model  has  been  modified  to  exclude,  as  a candidate  firing  position, 
any  hover  fire  position  that  is  located  over  an  area  of  the  battlefield  in  which  a 
knap  of  the  earth  flight  profile  would  be  exposed  to  any  enemy  weapon. 

Second,  a procedure  has  been  adopted  to  account  for  the  increased  expo- 
sure that  may  result  from  a highly  elevated  firing  position.  The  procedure 
assumes  that  the  firing  activity  segment  is  exposed  to  all  enemy  weapons  on  the 
battlefield  and  that  the  transition  segments  are  conducted  at  tree  top  level. 

Thus,  the  basic  procedure  remains  unchanged.  However,  two  increments  of 
time  are  added  to  the  exposure  time  associated  with  the  firing  activity  route 
segment.  The  first  increment  is  intended  to  account  for  the  time  that  would 
be  required  to  climb  to  the  firing  position  from  a knap  of  the  earth  flight  level. 

The  second  increment  is  intended  to  account  for  the  time  that  would  be  required 
to  dive  from  the  firing  segment  back  to  a knap  of  the  earth  flight  level.  Thus, 
the  perceived  threat  computed  for  the  firing  activity  segment  effectively  com- 
prehends  increases  in  threat  arising  because  of  exposed  climbs  to  and  dives  from 
a firing  position. 

The  procedure  used  to  compute  the  exposure  increments  is  as  follows : 

1.  Determine  the  elevation  of  the  firing  (exit)  position. 

2.  Determine  the  elevation  of  a knap  of  the  earth  flight  level  at  the 
firing  (exit)  position. 
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3.  Determine  the  climb  (descent)  required. 

4.  Compute  the  exposure  time  increment  to  be  added  based  on  a 
standard  military  rate  of  climb. 

The  reader  will  note  that  nothing  has  been  done  to  force  the  selection  of 
a firing  position  at  the  maximum  effective  range  of  the  weapon.  Except  for  the 
points  excluded  as  a result  of  the  procedure  outlined  in  a previous  paragraph, 
the  candidate  firing  positions  are  determined  as  before. 

The  reason  that  this  step  has  not  been  taken  is  that  the  model  is  still 
based  upon  the  following  fundamental  proposition:  From  a set  of  all  feasible 
route  alternatives,  the  decision  maker  chooses  that  route  which  possesses 
minimum  perceived  threat.  The  set  of  alternatives  originally  considered 
feasible  by  the  model  has  been  reduced  to  some  degree  by  ruling  out  hover 
fire  positions  that  involve  exposed  approaches  to  these  positions.  However, 
it  has  been  decided  to  reduce  the  feasible  set  no  further  until  additional  experi- 
ments can  be  conducted  to  determine  whether  or  not  such  reductions  are  required. 

Firing  Mode  Decisions.  It  has  been  stated  that  the  subjects  apparently  favor 
hover  fire  over  running  fire  unless  they  are  forced  into  a running  fire  duel 
with  the  target  by  some  factor  such  as  the  terrain  (lack  of  covered  approaches) 
or  weapon  effective  range  (their  own  versus  the  enemy's).  This  conclusion  is 
formed  largely  from  the  statements  sumrru  rized  in  Tables  6 and  7 and  Exhibit 
F.l. 

However,  there  are  probably  additional  factors  which  also  influence  the 
decision.  For  example,  it  is  quite  likely  tliat  individual  style  and  experience 
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are  important  factors.  In  fact,  considering  previous  results,  it  would  be  very 
surprising  to  find  that  the  population  of  decision  makers  is  homogeneous  in  the  , 

way  it  formulates  the  mode -of -fire  decision.  One  subject  even  stated  that  he 
had  had  no  occasion  in  combat  to  select  one  mode  of  fire  over  another. 

Unfortunately,  there  are  no  data  available  with  which  to  perform  precise 
statistical  analyses  of  the  factors  influencing  the  mode-of-fire  decision.  The 
factorials  employed  in  the  Phase  II  Experiment  were  developed  to  permit 
analyses  of  the  factors  affecting  the  overall  predictive  performance  of  the  route 
selection  model.  In  particular,  the  analyses  were  directed  at  evaluating  the 
three  proposed  threat  perception  models.  The  mode-of-fire  decision  process 
was  not  a specific  concern  of  the  experiment. 

However,  there  are  some  quantitative  data  available  which  may  be 
analyzed  to  a degree.  Furthermore,  they  tend  to  confirm  the  general  conclusions 
stated  earlier  which  were  formed  on  the  basis  of  qualitative  statements  advanced 
by  the  subjects. 

Table  8 reveals  the  mode  of  fire  selected  in  each  route  selection  situation 
analyzed  by  the  subjects.  In  addition,  the  target  weapon,  the  supporting  weapon, 
their  position  numbers  (see  Appendix  A),  and  the  intervisibility  condition  (with 
respect  to  the  target  weapon)  that  existed  at  the  start  of  the  route  are  also 
■ shown . 

First,  it  may  be  seen  that  out  of  the  twenty-four  cases,  the  subjects 


selected  hover  fire  fifteen  times  (62%).  While  this  is  not  a particularly  high 
f mm ir> nr v . tho  fact  that  it  is  creator  than  fiftv  percent  still  could  be  construed 
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Table  8 


Mode  of  Fire  Selection  Data 


Legend 


Weapon  Type 


1 - 14.5  mm 
2-23  mm 
3 - Redeye 


Mode 
of  Fire 
Selected 


Initial  Intervisibility 
Condition 


U - unexposed  to  target  weapon 
E - exposed  to  target  weapon 


Mode  of  Fire  Selected 


H - hover  fire 
R - running  lire 
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as  evidence  of  a preference  for  hover  fire.  Unfortunately,  the  probability  of 
achieving  at  least  this  frequency  even  if  the  choice  of  firing  mode  were  random 
is  0.20.  Thus,  the  evidence  is  not  statistically  overwhelming. 

Now , there  are  no  data  in  Table  8 to  precisely  reveal  the  effects  that 
terrain  and  weapon  effective  range  have  on  the  choice  of  firing  mode . Each 
situation  is  very  complex,  involving  two  weapons  with  different  effective  ranges 
located  at  two  distinct  positions.  Furthermore,  a high  degree  of  confounding 
exists  with  respect  to  the  weapons  and  positions  involved  as  the  subject  and/ or 
the  situation  factor  level  changes.  However,  an  indication  that  terrain  and 
weapon  effective  range  do  affect  the  choice  of  firing  mode  is  evident  in  the  data. 
For  example,  in  situations  involving  a 14.5  mm  target  weapon , seventy  percent 
of  the  cases  represent  hover  fire  choices  (7  out  of  10  cases).  On  the  other  hand, 
routes  directed  at  a 23  mm  target  weapon  result  in  hover  fire  only  fifty-seven 
percent  of  the  time  (8  out  of  14  cases).  If  the  analysis  is  further  restricted  to 
consider  only  those  routes  that  start  from  an  exposed  position,  the  incidence  of 
hover  fire  increases  to  eightly  percent  (4  out  of  5 cases)  when  the  target  is  a 
14.5  mm  weapon,  and  decreases  to  forty -three  percent  (3  out  of  7 cases)  when 
the  target  is  a 23  mm  weapon.  An  explanation  for  these  data  trends  lies  in  the 
fact  that  a 14.5  mm  weapon  has  an  effective  range  of  1400  meters  as  opposed  to 
a 3000  meter  range  for  the  23  mm  weapon.  The  subjects  evidently  felt  they 
could  deliver  fire  from  a hover  fire  position  outside  the  range  of  the  14.5  mm 
weapon  more  often  than  they  could  against  a 23  mm  weapon. 
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Evidence  of  heterogeneity  among  the  subjects  is  also  available  from  the 
data  in  Table  8.  First,  the  reader  will  recall  that  the  subject  pairs  (7,  11)  and 
(8,  12)  were  confronted  with  identical  route  selection  situations.  Yet,  in  both 


cases  the  subject  pairs  disagreed  on  the  mode  of  fire  to  be  employed  in  fifty 
percent  of  the  situations.  Furthermore,  subject  8 chose  running  fire  in  all 
situations.  Such  a performance  appears  to  be  different  from  that  of  the  other 
subjects. 

Firing  Mode  Prediction.  Given  the  observations  in  the  preceding  paragraphs 
and  considering  previous  conclusions  based  on  qualitative  evidence,  it  is 
obvious  that  there  are  factors  such  as  terrain,  weapon  effective  range  and 
individual  subject  performance  that  do  affect  the  choice  of  a mode  of  fire.  Thus, 
an  experiment  could  be  designed  to  yield  data  on  these  relationships.  However, 
for  our  purposes  it  is  not  clear  that  such  an  experiment  would  be  entirely  jus- 
tified. That  is,  it  may  not  be  so  important  that  detailed  knowledge  of  the 
mode-of-fire  decision  process  be  acquired.  Rather,  it  is  probably  more 
important  that  it  be  possible  to  use  the  route  selection  model  to  predict  the  mode 
of  fire  that  a decision  maker  would  employ  in  a given  situation.  Therefore,  in 
the  following  paragraphs  a procedure  is  proposed  that  may  produce  the  desired 
result.  The  validity  of  the  precedure  is  testable,  and  in  fact,  has  been  tested 
as  will  be  explained  in  the  analysis  section  of  this  chapter. 

The  reader  will  recall  that  the  route  selection  model  is  designed  to 
determine  a route  that  includes  cither  a hover  lire  or  a running  fire  attack  upon 
the  target.  The  type  of  attack  to  lx'  included  is  specified  beforehand. 
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Furthermore , any  route  that  is  selected  has  the  property  of  minimum  perceived 
threat,  where  threat  is  as  measured  by  one  of  the  three  threat  perception  models 
developed  in  Chapter  3. 

Now,  the  procedure  proposed  for  using  the  route  selection  model  to  pre- 

' 

diet  the  mode-of -fire decision  in  a particular  tactical  situation  is  as  follows: 

Using  a given  threat  perception  model,  determine  a route  that  includes  a hover 
fire  attack.  Denote  the  overall  threat  of  the  route  as  Tpjp.  Repeat  the  analysis 
for  a running  fire  route  to  obtain  T^p.  If  T^p  < Tj^p,  then  predict  that  the 
decision  maker  will  prefer  the  hover  fire  route  result.  Otherwise,  pre- 
dict that  the  decision  maker  will  prefer  the  running  fire  route  result. 

Obviously,  a major  assumption  of  the  above  procedure  is  that  the  decision 
maker  is  guided  in  his  mode-of-fire  decision  by  the  same  set  of  principles  that 
he  uses  in  selecting  other  portions  of  his  route.  Translated  into  terms  used  to 
develop  the  route  selection  model,  the  decision  maker  simply  acts  so  as  to 
minimize  perceived  threat.  Thus,  it  is  necessary  only  to  determine  the  per- 
ceived threat  of  two  alternative  courses  of  action.  The  alternative  that  is  selec- 
ted is  the  one  that  minimizes  perceived  threat.  Of  course,  it  is  still  assumed 
that  the  procedures  used  to  evaluate  the  perceived  threat  of  other  portions  of  the 
route  can  be  used  in  evaluating  the  perceived  threat  of  the  attack  phase.  This 
assumption  has  applied  without  exception  throughout  the  entire  research  pro- 
gram. If  specialized  procedures  are  required,  then  the  modeling  approach 
needs  to  be  revised.  Further  experimentation  will  yield  information  regarding 
this  possibility. 
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There  are  several  difficulties  that  may  arise  in  applying  the  mode-of-fire 
decision  procedure  that  has  been  outlined.  These  difficulties  should  be  dis- 
cussed at  this  point  to  avoid  problems  later.  First,  it  is  possible  that  incon- 
sistent results  may  be  obtained  from  the  three  threat  perception  models.  That 
is,  the  procedure  might  predict  that  the  hover  fire  route  result  is  preferable  to 
the  decision  maker  when  threat  is  measured  by  one  of  the  three  threat  models, 
but  that  the  running  fire  route  result  is  preferable  when  another  threat  model  is 
used.  Therefore,  it  is  proposed  that  the  procedure  can  be  applied  with  total 
validity  only  after  the  threat  perception  model  which  applies  to  a given  decision 
maker  has  been  determined.  Furthermore,  the  true  threat  perception  model 
for  a particular  decision  maker  may  not  even  be  one  of  the  three  models  under 
consideration. 

Another  problem  lies  in  the  fact  that  the  procedure  does  not  predict  with 
certainty.  Even  if  it  were  known  with  certainty  that  the  route  selection  model 
structures  a route  by  exactly  the  same  procedures  used  by  a decision  maker, 
the  intrapersonal  variation  of  the  decision  maker  might  yield  different  rankings 
for  the  route  selection  model  results  in  repeated  trials  of  the  same  decision 
problem.  Thus,  the  procedure  can  predict  only  the  most  probable  ranking  of 
the  model  results. 

Finally,  another  problem  related  to  the  one  just  outlined  is  that  the  pro- 
cedure may  not  predict  the  mode  of  fire  that  a decision  maker  actually  selects  in 
a given  situation.  Even  if  the  model  were  to  perfectly  duplicate  the  processes 


173 


involved  in  the  mode-of-fire  decision,  the  intrapersonal  variation  of  the  deci- 
sion maker  would  rule  out  prediction  with  certainty.  The  procedure  could  pre- 
dict only  the  most  probable  firing  mode  decision. 

To  compound  the  problem  still  further,  there  remains  a considerable 
amount  of  uncertainty  about  the  model  itself.  To  this  point  in  time  there  is  no 
guarantee  that  the  route  selection  model  perfectly  duplicates  the  decision 
processes  of  the  decision  maker.  Any  disparity  that  exists  could  reduce  the 
effectiveness  of  the  proposed  mode-of-fire  prediction  procedure. 


A Note  on  Ratings.  The  thrust  of  this  section  has  been  to  isolate  those  factors 
which  may  have  contributed  to  a poorer  than  expected  acceptance  of  route  selec- 
tion model  results  during  the  Phase  II  Experiment.  Changes  to  the  model  have 
been  proposed  to  better  represent  the  selection  of  a firing  position,  and  a pro- 
cedure has  been  developed  to  predict  the  firing  mode  that  a decision-maker  will 
select  in  a given  situation. 

However,  it  is  suspected  that  another  factor  is  present  which  may  also 
contribute  to  the  apparently  poor  showing  of  the  route  selection  model.  This 
factor  could  be  more  important  than  any  other  factor  considered  so  far,  and  if 
it  is  present  it  might  be  extremely  difficult  to  deal  with.  It  is  suspected  that  the 


subjects  are  actually  biased  against  the  route  selection  model  results , either 
because  they  are  unable  to  accurately  judge  the  relative  "goodness"  of  a route 
alternative  or  because  they  subjectively  reject  a route  that  employs  a firing 
mode  different  from  the  mode  they  used  in  the  same  situation. 
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An  indication  of  this  latter  type  of  bias  is  reflected  by  the  data  in  Table 
9.  This  table  shows  the  mode  of  fire  selected  for  each  situation  faced  by  the 
subjects  during  the  Phase  II  Experiment.  Also  shown  are  the  average  ratings 
given  to  the  hover  fire  and  running  fire  routes  produced  by  the  route  selection 
model.  As  anyone  can  see,  some  of  the  subjects  apparently  gave  higher  ratings 
to  model  routes  if  they  happened  to  include  a mode  of  fire  that  corresponded  to 
the  mode  selected  by  the  subject.  Overall,  sixty-four  percent  of  the  ratings 
(14  out  of  22  cases)  reflect  this  tendency. 

Also,  the  data  reveal  that  the  subjects  are  not  necessarily  biased  against 
just  the  hover  fire  routes  or  just  the  running  fire  routes.  Such  a result  would 
not  be  indicative  of  bias.  Instead  it  would  indicate  some  shortcoming  in  the 
method  used  to  represent  hover  fire  or  running  fire. 

Now,  an  experimental  technique  required  to  reduce  the  type  of  bias  out- 
lined above  would  be  to  somehow  conceal  from  the  subject  the  identity  of  his 
own  route  during  an  evaluation  of  the  model  routes.  Another  technique  would 
be  to  develop  some  rating  procedure  for  the  route  selection  model  results  that 
does  not  involve  selection  of  routes  by  the  subjects.  However,  it  is  difficult  to 
see  how  the  performance  of  the  model  could  be  measured  unless  some  benchmark 
were  available  to  serve  as  a reference.  More  on  this  topic  will  appear  in  Chap- 
ter 6. 

The  other  type  of  bias  mentioned  above  may  also  affect  the  apparent 
acceptability  of  the  model  results.  This  bias  is  related  to  the  subjects'  ability 
to  actually  judge  the  relative  "goodness"  of  two  different  route  alternatives. 
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Table  9 


Indication  of  Bias 


Subject 

Situation 

Firing  Mode 
Selected* 

Average  Rating  j 

Running  Fire 

Hover  Fire 

6 

1 

H 

0 

0 

2 

H 

wmmt 

I H 

3 

H 

Kji^H 

mm 

4 

R 

mmm 

7 

1 

H 

0.82 

0.95 

2 

H 

0.63 

0.68 

3 

H 

0.94 

0.97 

4 

R 

0.99 

0.98 

8 

1 

R 

0.60 

0.46 

2 

R 

0.89 

0.85 

3 

R 

0.93 

0.92 

4 

R 

0.94 

0.89 

11 

1 

R 

0.72 

0.62 

2 

H 

0.51 

0.57 

3 

H 

0.90 

0.88 

4 

H 

0.70 

0.84 

12 

1 

H 

0.91 

0.57 

2 

H 

0.83 

0.61 

3 

R 

0.57 

0.38 

4 

R 

0.51 

0.57 

13 

1 

H 

0.52 

0.39 

2 

H 

0.33 

0.32 

3 

H 

0.80 

0.59 

4 

H 

0.47 

0.84 

* Hover  Fire  - H 
Running  Fire  - R 
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For  example,  a subject  might  feel  that  a particular  firing  position  is  rather 
poor  because  of  the  amount  of  exposure  time  that  would  be  required  in  firing 
from  the  position.  Yet,  detailed  analyses  could  reveal  that  the  actual  amount 


of  exposure  time  would  be  less  for  the  route  in  question  than  it  would  be  for 

some  other  route  judged  to  be  superior.  Perceptual  problems  such  as  these 

became  evident  in  the  comments  recorded  by  the  subjects  during  the  route 

evaluation  session  of  the  Phase  II  Experiment  (see  Table  6). 

To  alleviate  perceptual  difficulties,  it  is  possible  to  develop  a large 

number  of  quantitative  descriptors  of  any  route  selected  by  the  route  selection 

model.  For  example  one  might  compute  the  following  data: 

Total  route  length 
Total  travel  time 
Total  exposure  time 

Range  to  the  target  from  the  firing  point 
Range  to  the  target  at  the  time  of  missile  impact 
Altitude  of  the  ^ring  position 
Exposure  time  during  the  attack 

Such  data  could  then  be  used  in  conjunction  with  a sketch  of  the  route  to  formu- 
late a more  valid  opinion.  In  fact,  as  will  be  explained  in  subsequent  sections 
of  this  chapter,  the  route  selection  model  has  been  revised  to  prepare  data 
such  as  those  outlined  above,  and  subjects  have  been  given  the  data  to  aid  in 
evaluating  route  alternatives. 

The  net  result  of  bias  is  that  the  route  selection  model  may  produce 
routes  that  are  actually  more  acceptable  to  the  subjects  overall  than  is  indicated 
by  the  Phase  II  Experiment  rating  data.  In  evaluating  the  validity  of  the  model  one 
must  be  very  careful  to  correct  for  this  bias  during  analyses,  or  to  revise  the 
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experimental  procedure  to  eliminate  it  in  the  first  place.  The  characteristics 
of  some  of  these  procedural  revisions  have  been  outlined  in  previous  paragraphs 
and  will  be  discussed  again  later  in  this  chapter  and  in  Chapter  6. 

The  Trial  Two  Experiment 

Given  the  results  of  Chapter  4 and  the  findings  of  the  previous  section, 
it  is  obvious  that  there  is  a need  for  further  experimental  investigations  into  the 
route  selection  decision  process.  First,  there  is  a need  to  determine  whether 
the  modifications  to  the  fire  position  selection  procedure  have  had  any  effect  on 
the  overall  acceptability  of  the  route  selection  model  results.  Second,  there 
is  a need  to  determine  whether  additional  indications  of  population  heterogeneity, 
similar  to  those  discovered  in  the  Phase  II  Experiment,  can  be  obtained.  Finally, 
there  is  a need  to  determine  whether  or  not  the  proposed  procedure  for  pre- 
dicting the  mode-of-fire  decision  is  valid.  Throughout  such  experiments , it  is 
desirable  that  procedures  be  employed  to  eliminate  as  much  of  the  bias  discussed 
in  the  preceding  section  as  possible,  subject  to  time  andfunds  available. 

To  this  end,  another  experiment  has  been  conducted  which  was  similar 
in  content  and  approach  to  the  Phase  II  Experiment.  In  fact,  the  subject  popu- 
lation used  was  the  same  population  employed  during  the  Phase  II  Experiment, 
with  the  exception  of  subjects  9 and  10  whose  data  were  not  analyzed  for  the 
reasons  discussed  in  Chapter  4.  The  experiment  will  be  called  the  Trial  Two 
Experiment  in  the  remainder  of  this  chapter. 
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Because  of  time  and  funds  available,  the  Trial  Two  Experiment  had 
to  be  conducted  by  mail.  This  meant  that  the  experiment  was  constrained  to 
use  the  subject  group  employed  during  the  Phase  II  Experiment  since  it  would 
have  been  almost  impossible  to  familiarize  new  subjects  with  the  concepts 
involved  without  personal  contact.  Also,  the  Trial  Two  Experiment  had  to  be 
kept  simple,  with  as  few  departures  as  possible  from  the  procedures  used 
during  Phase  II.  This  constraint  was  necessary  because  of  the  reduced  ability  to 
introduce  new  concepts  and  to  communicate  intricate  instructions  via  the  mails. 
Finally,  it  was  considered  desirable  to  keep  the  experiment  as  small  as  possible 
in  order  to  assure  that  the  subjects  would  cooperate  in  completing  the  experi- 
ment in  a reasonable  amount  of  time . 

The  net  result  of  the  constraints  outlined  above  is  that  it  was  impossible 
to  achieve  all  of  the  desired  objectives  with  the  Trial  Two  E'  ^'iment.  First, 
in  reducing  the  size  of  the  experiment,  not  all  the  data  frv  Phase  II 

Experiment  were  reproduced  by  the  Trial  Two  Experiment.  Not  only  did  this 
prohibit  a direct  comparison  of  all  data  from  the  two  experiments,  but  it  also 
reduced  the  amount  of  information  produced  by  the  Trial  Two  Experiment. 

Next,  it  was  considered  impossible  to  introduce  new  procedures  that 
wauld  entirely  eliminate  the  subject  bias  discussed  previously.  While  it  was 
possible  to  modify  the  route  selection  model  to  produce  quantitative  descriptors 
of  each  route  to  be  used  in  conjunction  with  a route  sketch  during  the  route 
evaluation  exercise,  it  was  beyond  the  scope  of  the  experiment  to  institute 
procedures  that  would  conceal  the  identity  of  a subject's  route  or  otherwise 
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eliminate  the  need  for  a reference  route.  Thus,  procedures  to  eliminate  only 
a portion  of  the  possible  subject  bias  were  available. 

Experimental  Design.  With  the  exception  of  size,  the  design  of  the  Trial  Two 
Experiment  was  almost  identical  to  that  of  the  Phase  II  Experiment.  There- 
fore, only  a summary  of  the  new  design  will  be  presented.  For  a more  thorough 
explanation  of  the  design  the  reader  should  consult  Chapter  4. 

The  overall  Trial  Two  experimental  design  consisted  of  four  independent 
complete  factorial  experiments,  of  which  two  were  four  factor  experiments  and 
two  were  three  factor  experiments.  The  previous  design  had  consisted  of  two  five 
factor  experiments  and  two  four  factor  experiments.  However,  a blocking  factor 
was  eliminated  from  all  four  factorials  for  the  Trial  Two  Experiment.  This  factor 
had  been  included  originally  to  permit  an  analysis  of  the  effect  that  input  data  in 
the  form  of  population  averages  would  have  on  the  acceptability  of  route  selection 
model  results.  It  was  found  in  Chapter  4 that  one  should  always  use  input  data 
that  apply  to  an  individual  decision  maker  as  opposed  to  population  averages. 

Now,  all  of  the  factorials  included  a qualitative  factor  M,  measured 
at  m = 3 fixed  levels,  to  account  for  a threat  perception  model  effect.  In  this 
respect  the  Trial  Two  Experiment. was  identical  to  the  Phase  II  Experiment. 

Of  course,  the  three  models  considered  were  the  Acid  Test  Model,  Model  One 
and  Model  Three  as  defined  previously. 

Another  factor  included  in  all  factorials  was  the  qualitative  faetor  R 
to  account  for  a route  (or  situation)  effect.  While  the  Phase  II  Experiment  had 
measured  this  factor  at  r = 4 fixed  levels,  the  Trial  Two  Experiment  considered 

180 


only  two  fixed  levels . This  change  was  adopted  as  part  of  the  effort  to 
reduce  the  size  of  the  experiment. 

The  situations  left  in  the  experiment  were  those  that  have  been 
referred  to  as  situation  one  and  situation  three.  From  Table  8 one  may  deter- 
mine that  these  situations  are  the  ones  in  which  the  starting  point  of  the  route 
is  not  exposed  to  the  target  weapon.  It  was  reasoned  that  if  any  situations 
were  to  be  eliminated,  it  would  be  best  to  eliminate  situations  two  and  four. 

They  were  considered  artificial  by  the  subjects  since  no  pilot  would  select  a 
route  prior  to  entry  into  the  battlefield  that  would  result  in  an  exposed  initial 
position.  However,  elimination  of  the  two  routes  did  reduce  the  information 
produced  by  the  Trial  Two  Experiment. 

The  final  factor  that  appeared  in  all  factorials  was  the  qualitative 
attack  mode  factor  A,  measured  at  a = 2 fixed  levels.  Thus,  the  Trial  Two 
Experiment  was  identical  to  the  Phase  II  Experiment  in  this  respect.  Of  course, 
the  two  attack  modes  considered  were  running  fire  and  hover  fire. 

Now,  there  was  a final  qualitative  factor  S which  appeared  in  two  of 
the  factorials  but  which  was  absent  from  the  other  two.  This  factor  was 
included  to  account  for  inter-subject  variability  and  was  measured  at  s = 2 levels. 
Furthermore,  the  subject  effect  was  considered  random. 

The  reader  will  recall  that  it  had  been  intended  originally  to  include 
the  subject  factor  in  all  four  of  the  Phase  II  Experiment  factorials.  In  fact, 
four  subject  pairs,  (G,  10),  (7,  11),  (8,  12)  and  (9,  13)  were  actually  processed 
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through  the  entire  Phase  II  Experiment.  However,  it  became  necessary,  as 
explained  in  Chapter  4,  to  exclude  from  analysis  the  data  collected  from  sub- 


jects 9 and  10.  For  the  same  reasons,  subjects  9 and  10  were  excluded  from 
the  Trial  Two  Experiment.  Therefore,  both  the  Phase  II  and  the  Trial  Two 
Experiments  were  identical  in  their  treatment  of  a subject  effect. 

To  summarize  the  preceding  discussion,  the  Trial  Two  experimental 
design  consisted  of  four  independent  complete  factorials  consisting  of  two 
(s)  x (r)  x (m)  x (a)  factorials  and  two  (r)  x (m)  x (a)  factorials , where  the 
symbols  are  as  defined  in  previous  paragraphs.  Each  of  the  four  factor  experi- 
ments required  a total  of  24  observations  while  the  three  factor  experiments 
each  required  12  observations.  This  resulted  in  a grand  total  of  72  observa- 
tions, which  represents  a sizeable  reduction  from  the  384  observations 
involved  in  the  Phase  II  Experiment. 

For  analysis  purposes , a model  was  assumed  that  included  all  main 
effects  and  all  two  factor  interactions.  However,  all  three  factor  and  higher 
order  interactions  were  assumed  to  be  zero.  Furthermore,  all  four  factors 
were  considered  qualitative,  and  three  out  of  the  four  were  assumed  to  pro- 
duce fixed  effects.  Thus,  the  model  associated  with  the  Trial  Two  Experi- 
ment is  consistent  with  that  of  the  Phase  II  Experiment. 

Conduct  of  the  Experiment.  Much  of  the  groundwork  for  the  Trial  Two  Experi- 
ment had  been  done  during  the  Phase  I and  Phase  H experimental  exercises. 
First,  it  was  intended  that  the  routes  which  the  subjects  had  selected  during 
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Phase  II  continue  to  be  used  as  reference  routes  in  evaluating  route  selection 


model  results.  Second,  the  threat  perception  data  collected  during  Phase  I 
still  formed  a valid  input  data  set  for  the  route  selection  model.  The  only 
new  effort  involved  in  conducting  the  Trial  Two  Experiment  consisted  of  the 
following  activities: 

1.  Revision  of  the  route  selection  model  to  incorporate  modifications 
to  the  fire  position  selection  procedures; 

2.  Revision  of  the  route  selection  model  so  that  it  would  prepare  a 
set  of  quantitative  descriptors  for  each  route  selected; 

3.  Execution  of  the  experimental  design  with  the  route  selection 
model;  i.e.,  preparation  of  a route  description  for  each  com- 
bination of  factor  levels  considered  in  the  experiment;  and 

4.  Preparation  of  a mailer  for  each  subject  participating  in  the 
experiment. 

Each  of  these  activities  will  be  discussed  in  the  following  paragraphs. 

From  a previous  discussion,  the  reader  will  recall  that  the  fire  position 
selection  procedures  were  modified  to  exclude  situations  in  which  the  approach 
to  a hover  fire  position  is  exposed  to  the  enemy.  The  procedures  were  also 
modified  to  account  for  increased  exposure  time  resulting  from  climbs  to 
elevated  firing  positions.  However,  changes  to  the  logic  of  the  model  to 
incorporate  these  modifications  were  minimal.  By  far  the  more  extensive 
changes  were  those  required  to  produce  a set  of  quantitative  descriptors  for 
each  route  selected.  Extensive  bookkeeping  procedures  were  required  to 
accumulate  some  of  the  data  which  were  desired  as  output. 
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The  reader  will  recall  that  the  descriptors  were  desired  so  that  a 


subject  asked  to  evaluate  a given  route  alternative  might  have  available  as 
much  objective  data  as  possible.  It  was  hoped  that  these  data  would  alleviate 
problems  the  subjects  apparently  experience  in  accurately  assessing  the 
characteristics  of  a route.  Consequently,  the  model  was  modified  to  produce 
the  following  descriptors: 

1.  Total  route  length 

2.  Total  travel  time 

3.  Total  exposure  time  to  the  target  weapon  only 

4.  Total  exposure  time  to  the  supporting  weapon  only 


5 . Total  exposure  time  to  both  weapons  simultaneously 

6 . Altitude  of  terrain  at  the  target 

7.  Range  to  the  target  from  the  firing  position 

8.  Altitude  of  the  helicopter  above  the  terrain  at  the  firing 
position 

9.  Missile  time  of  flight 

10.  Range  to  the  target  at  time  of  missile  impact 

11.  Altitude  of  the  helicopter  above  the  terrain  at  the  time 
of  missile  impact 

12.  Dive  angle  (climb  angle)  for  running  fire 

13.  Exposure  time  for  the  firing  event  (climb,  aiming  delay, 
missile  flight,  dive) 

14.  Probability  of  hitting  target 
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After  all  model  modifications  had  been  made  the  next  major  step  was 


execution  of  the  experimental  design  with  the  route  selection  model.  This 
required  that  the  model  be  used  to  prepare  a route  description  for  each  com- 
bination of  factor  levels  considered  in  the  experiment.  Thus,  a total  of  72 
route  descriptions  were  prepared. 

As  was  done  during  the  Phase  II  Experiment,  route  descriptions  pre- 
pared by  the  model  were  collected  and  then  plotted  in  groups  of  three  to  a 
page.  For  each  subject-situation  combination  the  three  running  fire  alterna- 
tives were  plotted  on  one  page, and  the  three  hover  fire  alternatives  were  plotted 
on  another.  Each  route  sketch  was  annotated  with  enough  information  to  deter- 
mine the  d of  travel  at  all  points  and  the  location  of  the  firing  position. 

Each  sk  also  identified  with  an  identification  number  to  be  used  by  the 

subject  during  the  route  evaluation  exercise.  This  same  number  appeared  on 
a data  sheet  containing  all  the  route  descriptor  data  described  earlier. 

Now,  a new  feature  of  the  Trial  Two  Experiment,  a sketch  and  a 
statistic  sheet7were  also  prepared  for  the  route  that  the  subject  had  chosen 
himself  during  Phase  n.  It  was  reasoned  that  the  same  amount  of  information 
should  be  available  for  all  routes  being  considered  by  the  subject,  and  of  course, 
the  subject's  own  route  was  to  be  used  as  a reference  during  the  evaluation 
exercise. 

The  final  step  in  conducting  the  experiment  was  the  preparation  of  a 
mailer  for  each  subject  participating  in  the  experiment.  An  example  of  the 
materials  contained  in  this  mailer  appears  in  Appendix  F as  Exhibit  F.2. 
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Exhibit  F.2  is  actually  a copy  of  the  package  received  by  subject  7.  However, 
some  of  the  materials  have  been  reduced  from  their  original  size  of  7-1/2x14-1/2. 
Moreover,  the  color  coding  originally  used  in  the  route  sketches  has  been  lost 


in  the  reduction  (reproduction)  process. 

The  packages  were  mailed  on  28  February  1972,  and  as  indicated  by  the 
cover  letter  in  each  package  (Exhibit  F.2),  the  subjects  were  urged  to  complete 
the  exercise  by  8 March  1972.  To  stress  the  importance  of  the  material,  each 
package  was  sent  via  certified  air  mail  with  an  air  mail  return  envelope 
enclosed.  Four  of  the  six  subjects  subsequently  met  this  deadline.  However, 
the  remaining  two  subjects,  for  various  reasons,  failed  to  respond  until  24 
March  1972.  Telephone  calls  verified  that  these  subjects  were  late  in  responding 
only  because  of  extenuating  circumstances.  Thus,  the  response  to  the  Trial 
Two  Experiment  was  much  better  than  the  response  obtained  in  the  earlier 
experiment  conducted  by  mail  (see  Appendix  G). 

Analysis  and  Conclusions 

As  a first  step  in  the  analysis  of  results,  an  analysis  of  variance  was 
performed  upon  the  data  obtained  during  the  Trial  Two  Experiment.  This 
analysis  had  the  objective  of  determining  whether  or  not  the  model  modifications 
had  had  any  effect  upon  the  overall  acceptability  of  routes  produced  by  the 
route  selection  model.  Furthermore,  it  was  desired  to  determine  whether  a 
further  indication  of  population  heterogeneity  would  be  obtained  from  the  Trial 
Two  Experiment. 
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Another  objective  of  the  overall  analysis  was  to  determine  whether  the 
quantitative  descriptors  prepared  for  each  route  alternative  had  aided  the 
subjects  in  evaluating  the  alternatives.  In  essence  it  was  hoped  that  these 
descriptors  would  permit  the  subjects  to  render  a more  valid  judgement  of  the 
relative  merits  of  each  alternative. 

A final  objective  of  the  analysis  was  to  test  the  validity  of  the  procedure 
proposed  in  a previous  section  for  predicting  the  mode  of  fire  that  a subject  will 
choose  in  a given  tactical  situation. 

Analysis  of  Variance.  The  reader  will  recall  that  the  design  employed  during 
Trial  Two  was  almost  identical  to  that  of  the  Phase  II  Experiment.  Therefore, 
the  two  analyses  of  variance  were  similar.  The  only  major  difference  was  that 
there  was  no  longer  an  input  data  set  effect  to  be  analyzed  from  the  Trial  Two 
Experiment. 

Another  point  of  similarity  between  the  two  analyses  was  the  procedure 
required  to  account  for  missing  data.  As  discussed  at  length  in  Chapter  4, 
different  factor  level  combinations  did  not  always  produce  unique  alternatives 
for  the  subjects  to  evaluate  during  the  Phase  II  Experiment.  Thus,  the  intended 
complete  factorials  degenerated  into  incomplete  factorials.  The  same 
phenomenon  occurred  in  the  Trial  Two  Experiment. 

In  the  interest  of  brevity  suffice  it  to  state  that  the  methods  used  to 
account  for  missing  data  in  the  Phase  II  analyses  were  again  employed  during 
Trial  Two  analyses.  The  effect  of  this  action  is  that  analyses  of  variance 
performed  upon  the  data  arc  only  approximate,  and  one  must  be  very  careful 
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in  his  interpretation  of  the  results.  However,  the  intent  of  analyses  in  this 
section  is  primarily  to  compare  results  between  the  two  experiments. 
Consequently,  it  seems  reasonable  to  employ  similar  methods  even  if  they  are 
only  approximate.  Nevertheless,  if  the  reader  desires  to  perform  an  in  depth 
analysis  of  the  type  discussed  in  Chapter  4,  data  required  for  the  analysis  are 
contained  in  Tables  F.l  through  F.3  of  Appendix  F. 

The  results  of  the  Trial  Two  Experiment,  as  they  appear  a ~er  applica- 
tion of  the  missing  data  procedure,  are  presented  in  Table  F.4.  A series  of 
analyses  of  variance  have  been  performed  upon  these  data,  and  the  significant 
results  are  summarized  in  Figures  21  through  24.  Detailed  analysis  of  variance 
tables  corresponding  to  this  summary  appear  in  Appendix  F.  Also,  plots  of 
marginal  means  are  presented  in  Appendix  F to  indicate  trends  corresponding 
to  most  of  the  significant  results. 

Block  Effect 

In  Figure  21  the  reader  will  note  that  an  analysis  of  variance  has  been 
performed  in  which  data  from  the  Phase  II  Experiment,  corresponding  to  data 
collected  during  the  Trial  Two  Experiment,  have  been  used  as  though  they  were 
a second  replication  of  the  Trial  Two  Experiment.  A block  effect  has  been 
included  in  the  statistical  model  to  differentiate  between  the  two  experiments. 

It  may  be  noticed  that  a significant  block  effect  exists  for  only  one 
subject.  Moreover,  when  subject  pairs  are  analyzed  (Figure  23),  no  significant 
block  effect  is  present.  Thus,  it  would  appear  that  the  two  experiments  yielded 


* Significant  at  a = 0. 10 
**  Significant  at.  a = 0.05 
***  Significant  at  a = 0.01 


* Significant  at  a = 0.10 
**  Significant  at  a = 0.05 
***  Significant  at  a = 0.01 


Figure  22. — Summary  of  Significant  Results 
by  Subject  (Trial  Two  Only) 
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* Significant  at  a = 0.10 
**  Significant  at  a = 0.05 
***  Significant  at  a = 0.01 


Figure  23. — Summary  of  Significant  Results 
by  Subject  Pair 
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Figure  24 . — Summary  of  Significant  Results 
by  Subject  Pair  (Trial  Two  Only) 
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essentially  the  same  data  overall.  Consequently,  it  would  appear  that  the 
efforts  to  improve  the  performance  of  the  route  selection  model  have  not  been 
altogether  successful.  However,  it  is  too  early  in  the  analysis  to  form  a 
definite  conclusion  to  this  effect.  For  example,  examination  of  marginal  means 
in  Figure  F.17  reveals  that  for  two  subjects  there  is  an  indication,  though  not 
significant,  of  some  improvement.  Moreover,  subject  6 shows  a significantly 
higher  acceptance  of  model  results.  For  the  remaining  three  subjects  the 
model  declined  slightly  in  performance,  yet  none  of  the  decreases  were 
significant. 

As  another  example , it  is  seen  in  Figure  21  that  a significantly  different 
route  effect  was  present  in  the  two  experiments  for  three  of  the  subjects.  More- 
over, a significantly  different  perception  model  effect  was  present  for  two 
subjects.  Thus,  it  is  possible  that  the  route  selection  model  did  yield  better 
results  during  Trial  Two  under  certain  circumstances.  In  fact,  examination 
of  marginal  means  in  Figure  F.18  reveals  that  the  results  for  route  one  did 
improve  for  five  of  the  six  subjects  during  Trial  Two.  Moreover,  Figure  F.19 
shows  that  the  results  achieved  using  the  perception  model  known  as  Model 
One  during  Trial  Two  were  as  good  as  or  better  than  comparable  results 
achieved  during  Phase  II  for  every  subject.  More  on  this  topic  of  model 

improvement  will  appear  in  a later  section  of  this  chapter. 

Before  concluding  the  discussion  of  block  effect,  it  should  be  noticed  from 

Figures  21  through  24  and  the  comparable  figures  in  Chapter  4 (Figures  9 through 
12),  that  the  incidence  of  significant  results  ffom  the  Trial  Two  Experiment  is 

much  lower  than  from  the  Phase  if  Experiment.  The  result  is  that  the  Trial  Two 
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Experiment  by  itself  produced  very  little  in  the  way  of  statistically  meaningful 
results.  Trends  can  be  established  from  plots  of  marginal  means  and  com- 
pared  with  comparable  trends  from  Phase  II.  However,  there  is  significant 
evidence  for  only  a few  of  the  main  effects  and  interactions  included  in  the 
statistical  model. 

At  first  it  was  thought  that  perhaps  the  statistical  model  did  not  fit  the 
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data  collected  during  Trial  Two.  However,  Figure  25  shows  that  the  overall 
fit  was  as  good  as  or  better  than  that  achieved  during  Phase  n.  From  Chapter 
4 the  reader  will  recall  that  the  performance  measure  P is  the  ratio  of  sums  of 
squares  of  error  to  total  sums  of  squares  and  may  be  viewed  as  a measure  of 
the  degree  to  which  the  statistical  model  fits  the  data.  That  is,  with  little 
unexplained  variability  it  takes  less  factor  variability  to  produce  a significant 
result.  Yet,  while  the  overall  fit  was  comparable,  fewer  significant  results 
were  obtained  from  Trial  Two.  Thus,  lack  of  fit  is  rejected  as  a reason  for 
the  reduced  incidence  of  significant  results  during  Trial  Two.  It  may  be  noted, 
however,  that  the  significant  results  that  were  obtained  did  come  primarily  from 
subjects  12  and  13.  Such  a result  is  to  be  expected  from  Figure  25. 

Thus , in  the  final  analysis , one  is  left  to  conclude  that  the  incidence  of 
significant  results  in  Trial  Two  declined  for  one  of  two  reasons.  First,  it  is 
possible  that  the  subjects  actually  were  less  affected  by  changes  in  the  factors 
considered  during  Trial  Two.  However,  an  equally  plausible  explanation  is 
that  reductions  in  the  size  of  the  experiment  were  primarily  responsible  for 
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the  decline.  During  Phase  n analyses  comparable  to  those  of  the  Trial  Two 


analyses  (see  Figures  E.7  - E.18  and  Figures  F.7  - F.12),  six  degrees  of 
freedom  were  available  in  the  residual.  Only  two  degrees  of  freedom  were 
available  in  the  Trial  Two  analyses . Furthe  rmore , the  route  factor  degrees 
of  freedom  declined  from  three  to  one.  This  caused  the  degrees  of  freedom 
associated  with  interactions  involving  the  route  factor  also  to  decline  by  the 
same  ratio.  Thus,  the  size  of  the  Trial  Two  Experiment  is  a very  likely  cause 
of  the  reduced  incidence  of  significant  results.  In  fact,  repetition  of  the 
Phase  n analyses  to  include  only  route  one  and  route  three  data  yields  the 
significant  results  shown  in  Figure  26.  These  results  are  very  similar  to  the 
results  shown  in  Figure  22  as  far  as  the  incidence  of  significance  is  concerned. 
Route  Effect 

As  mentioned  earlier  three  of  the  six  subjects  displayed  a significantly 
different  route  effect  in  the  two  experiments.  Furthermore,  examination  of 
Figure  F.18  reveals  that  there  are  indications  of  this  interaction  for  five  of 
the  six  subjects.  In  these  five  cases,  the  subjects  apparently  felt  that  the 
route  selection  model  had  performed  better  in  constructing  route  one  alter- 
natives during  Trial  Two  than  it  had  during  Phase  II.  On  the  other  hand,  in 
four  of  the  five  cases  the  route  three  alternatives  were  judged  to  be  poorer 
during  Trial  Two. 

The  desired  result  is  that  there  be  no  route  effect  or  interactions 
involving  the  route  factor.  One  would  prefer  that  the  route  selection  model 
perform  equally  well  in  all  route  selection  situations.  The  analyses  reveal 
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that  there  may  have  been  some  improvement  in  this  direction.  First,  from 
Figure  22  it  is  seen  that  the  route  factor  significantly  interacts  with  another 
factor  for  only  one  subject.  Furthermore,  the  route  effect  is  significant  for 
only  two  of  the  six  subjects. 

Part  of  this  decline  in  significance  may  be  due  to  the  reduced  size  of 
the  Trial  Two  Experiment  as  explained  previously.  However,  Figure  F.18 
reveals  that  the  slope  of  the  marginal  mean  plot  for  the  route  factor  has  been 
decreased  in  magnitude  for  three  of  the  subjects.  Furthermore,  it  is  no  longer 
true  that  for  most  subjects  the  model  works  better  for  route  3 than  for  route  1. 

In  Chapter  4 it  was  pointed  out  that  the  subjects  might  be  correlating  their 
judgment  of  the  performance  of  the  model  with  their  perception  of  the  difficulty 
of  the  route  selection  situation.  Such  a correlation  no  longer  exists.  There- 
fore, the  conclusion  is  that  some  progress  has  been  made  in  reducing  the  effect 
that  the  route  selection  situation  has  on  model  performance.  Moreover,  the 
subjects  may  have  given  more  objective  evaluations  of  model  performance 
during  Trial  Two.  However,  still  more  data  are  required  before  final  conclusions 
can  be  reached  in  this  area. 

Attack  Mode 

Examination  of  Figures  21  through  24  reveals  that  the  attack  mode  effect 
was  insignificant  in  Trial  Two.  Furthermore,  this  factor  interacted  signifi- 
cantly with  only  one  other  factor  (route)  for  only  one  subject.  Thus,  one 
might  conclude  that  the  improvements  made  in  the  hover  fire  position  selection 


procedures  of  the  model  have  achieved  the  desired  result.  That  is,  the  model 
may  now  represent  hover  fire  and  running  fire  attacks  equally  well.  Such  a 
conclusion  is  also  supported  by  the  marginal  mean  plots  of  Figure  F.20.  How- 
ever, the  lack  of  significance  may  be  due  to  the  factors  cited  previously.  More- 
over, it  may  still  be  true  that  the  fire  position  selection  procedures  are  better 
in  some  route  selection  situations  than  in  others  as  suggested  by  the  significant 
route-firing  mode  interaction  for  subject  12  and  the  marginal  mean  plots  of 
Figure  F.  22.  The  attack  mode  effect  will  be  discussed  again  in  a later  section 
of  this  chapter. 

Perception  Model 

The  perception  model  effect  was  one  of  the  most  significant  effects  in 
the  Phase  II  Experiment.  This  result  was  highly  desirable  since  one  of  the 
objectives  of  the  experiment  was  to  determine  which  of  the  three  proposed 
perception  models  was  most  representative  of  the  way  in  which  subjects  per- 
ceive threat.  Unfortunately,  it  was  found  that  not  all  subjects  perferred  the 
same  model.  Furthermore,  different  models  performed  better  in  different 
route  selection  situations. 

In  the  Trial  Two  Experiment,  only  one  instance  of  a significant  percep- 
tion model  effect  was  found  as  indicated  in  Figure  22.  Moreover,  a significant 
subject-perception  model  interaction  was  found  for  only  one  of  the  two  subject 
pairs  as  indicated  in  Figure  24.  A perception  model-route  interaction  did  not 
exist  in  the  Tri;il  Two  Experiment. 
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Lack  of  significance  in  the  Trial  Two  results  may  be  due  to  the  reduced 
size  of  the  Trial  Two  Experiment  as  explained  earlier.  In  the  analyses  which 
included  a block  effect  to  represent  differences  between  Phase  H and  Trial  Two, 
significant  perception  model  effects  were  found  for  three  of  the  subjects  (Figure 
21).  However,  it  was  also  found  that  for  two  of  these  three  subjects  different 
perception  model  effects  were  present  in  the  two  experiments.  Thus,  it  is 
difficult  to  draw  definite  conclusions  from  a simple  analysis  of  significant 
results. 

A more  revealing  analysis  may  be  performed  by  examining  the  marginal 
mean  plots  of  Figure  F.  19.  It  is  seen  that  with  the  exception  of  subject  13,  the 
results  of  Trial  Two  are  very  similar  to  those  obtained  during  Phase  II.  For 
example,  there  still  appears  to  be  a difference  between  Model  One  and  the  other 
two  perception  models  taken  together,  although  this  difference  appears  to  be 
less  for  some  of  the  subjects.  Furthermore,  the  Acid  Test  Model  and  Model 
Three  continue  to  be  scored  closely  together  in  relation  to  Model  One.  Of 
course,  missing  data  still  accounts  for  much  of  this  phenomenon  as  explained 
in  Chapter  4. 

Another  point  of  similarity  between  results  of  the  two  experiments  is  the 
effect  of  the  interaction  between  the  route  factor  and  the  perception  model 
factor.  In  the  Phase  n Experiment  a significant  interaction  was  found,  but  it 
was  shown  that  the  interaction  did  not  affect  to  any  degree  the  choice  of  a per- 
ception model.  In  most  cases  a subject  preferring  a given  model  would  con- 
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tinuc  to  prefer  the  same  model  as  the  route  changed. 
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In  the  Trial  Two  Experiment  a very  similar  effect  was  found.  First, 
there  was  not  even  a significant  route-perception  model  interaction.  Further- 
more, Figure  F.21  shows  that  a subject  preferring  a given  model  almost  always 
continues  to  prefer  the  same  model  in  different  situations.  Finally,  a minimax 
analysis  similar  to  the  one  conducted  in  Chapter  4 shows  that  Model  One  con- 
tinues to  be  the  preferred  model  in  the  face  of  uncertainty  with  respect  to  the 
situation  in  which  the  model  would  be  applied.  Figure  27  shows  that  should 
Model  One  be  adopted  the  worst  result  would  be  an  average  rating  of  88%. 
However,  to  choose  the  Acid  Test  Model  could  yield  an  average  rating  of  65% 
while  a choice  of  Model  Three  could  yield  an  average  rating  as  low  as  62%. 


Perception  Model 

Figure  27. — Perception  Model-Route  Interaction  Across 
All  Subjects  (Trial  Two) 

201 


1 


From  the  above  observations  it  appears  that  the  Trial  Two  Experiment 
yielded  essentially  the  same  results  as  the  Phase  II  Experiment  with  respect 
to  perception  model  effect.  Thus , the  conclusions  reached  as  a result  of  Phase 
H are  unchanged  by  the  results  of  Trial  Two.  Model  One  is  apparently  preferred 
by  most  subjects  in  most  route  selection  situations.  In  the  face  of  uncertainty 
with  respect  to  the  situation  in  which  the  model  would  be  applied,  one  should 
choose  Model  One  to  be  used  in  the  route  selection  model  to  achieve  the  highest 
average  rating  across  all  subjects.  However,  from  Figure  F.19  it  is  obvious 
that  Model  One  is  not  the  model  preferred  by  all  subjects.  Again,  two  of  the 
six  subjects  preferred  a model  other  than  Model  One.  Thus,  it  would  be 
foolish  to  choose  Model  One  to  the  exclusion  of  all  other  models.  There  is  a 
35%  chance  that  some  other  model  would  be  the  choice  of  half  the  entire  Army 
pilot  population  possessing  the  backgrounds  of  the  subjects  tested.  Therefore, 
at  this  time  it  is  suggested  that  Model  One  be  chosen  as  the  most  promising 
candidate  of  the  three  proposed  perception  models.  However,  it  is  also  sug- 
gested that  further  experimentation  be  conducted  to  determine  if  alterations 
of  the  three  models  tested  can  produce  better  results.  Furthermore,  it  is 
desirable  to  test  other  subject  groups  with  backgrounds  different  from  the 
Phase  n subject  group  in  order  to  reduce  the  statistical  uncertainty  of  the 
results  achieved  so  far. 
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Subject  Effect 

In  Chapter  4 it  was  shown  that  the  subject  factor  interacted  frequently 
with  other  factors  in  the  Phase  II  Experiment.  Furthermore,  a strong  subject 
effect  occurred  in  most  analyses  of  variance,  although  the  main  effect  generally 
was  not  as  important  as  the  interactions.  This  conclusion  was  supported  by  the 
data  in  Figures  16  through  18. 

Figures  16  through  18  also  showed  that  subjects  8 and  12  exhibited  basic- 
ally two  different  modes  of  behavior.  The  marginal  mean  plots  of  Appendix  E 
indicate  that  subject  12  behaved  in  a fashion  similar  to  subject  13.  On  the 
other  hand,  subject  8 was  most  similar  to  subjects  6,  7,  and  11.  Thus,  it  was 
concluded  that  at  least  two  basic  modes  of  decision  behavior  may  have  existed 
within  the  subject  group. 

Now,  one  of  the  primary  objectives  of  the  Trial  Two  Experiment  was 
to  determine  whether  or  not  the  indication  of  population  heterogeneity  achieved 
during  Phase  II  could  be  duplicated  in  a second  experiment.  The  answer  is  an 
emphatic  yes. 

First  of  all,  even  though  analyses  of  variance  using  only  Trial  Two  data 
yielded  very  few  significant  results , it  is  seen  in  Figure  24  that  a significant 
subject  effect  was  present  in  both  factorials  involving  a subject  factor.  Further- 
more, for  subject  pair  (8,  12)  the  subject  factor  interacted  with  all  other  factors 
except  firing  mode. 
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Next,  Figure  28  shows  that  interactions  involving  the  subject  factor 
comprise  a significant  portion  of  the  overall  variability  attributable  to  inter- 
actions. Furthermore,  it  is  seen  that  the  variability  attributable  to  the  subject 
factor  for  subject  pair  (8,  12)  is  larger  than  the  variability  attributable  to  all 
the  other  factors  combined.  It  also  represents  a greater  contribution  to  overall 
variability  than  that  attributable  to  the  subject  factor  for  subject  pair  (7,  11). 
These  results  are  very  similar  to  those  obtained  in  Phase  H. 

Finally,  examination  of  the  marginal  mean  plots  in  Figure  F.19  reveals 
that  with  the  exception  of  subject  13,  all  subjects  exhibited  about  the  same 
behavior  in  rating  the  three  threat  perception  models  during  Trial  Two  as  they 
exhibited  during  Phase  II.  However,  during  Trial  Two  subject  13  changed  his 
ratings  significantly.  His  behavior  now  appears  to  be  very  similar  to  that 
exhibited  by  subjects  6,  7,  8,  and  11.  The  effect  of  this  change  is  that  subject 
12  now  appears  to  behave  in  a fashion  unlike  any  other  subject. 

Considering  the  above  results,  it  appears  reasonable  to  conclude  that 
there  is  a very  definite  subject  effect,  and  that  the  subject  factor  does  interact 
with  most  other  factors.  That  is,  the  overall  performance  of  the  route  selection 
model  is  judged  differently  by  different  subjects.  Moreover,  different  subjects 
feel  that  changes  in  the  route  selection  situation  and  the  perception  model  affect 
the  performance  of  the  route  selection  model  in  different  ways.  However,  it 
appears  that  the  population  of  Army  pilots  can  be  divided  into  two  groups, 
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Figure  28.- 

-Subject  Effects  and  Interactions  (Trial  Two  Data) 

model  and  the  other  group  containing  pilots  that  generally  prefer  some  other 
model.  Yet,  as  stated  previously,  more  experimentation  is  required  in  this 
area  before  final  conclusions  can  be  drawn.  For  example,  it  has  been  seen 
that  one  subject  (subject  13)  apparently  changed  his  mode  of  decision  behavior 
between  Phase  II  and  Trial  Two. 

Worth  of  the  Model.  As  seen  in  the  analyses  of  variance  presented  in  the  pre- 
ceding paragraphs , ratings  given  to  the  route  selection  model  by  the  subject 
group  vary  in  very  complex  ways  with  almost  every  factor  included  in  the 
Trial  Two  Experiment.  This  finding  is  consistent  with  that  of  the  Phase  II 
Experiment  reported  in  Chapter  4.  However,  a very  important  overall  ques- 
tion is  whether  or  not  the  changes  made  to  the  route  selection  model  in  prepara- 
tion for  th*>  Trial  Two  Experiment  had  any  effect  upon  the  performance  of  the 
model. 

In  a previous  section  it  was  shown  that  a significant  block  effect  between 
Phase  EL  and  Trial  Two  existed  for  only  one  subject.  Thus,  there  was  little 
indication  of  an  overall  improvement  in  model  performance.  However,  it  was 
also  pointed  out  that  the  model  may  have  performed  much  better  under  certain 
circumstances. 

As  an  approach  to  studying  this  possibility,  an  analysis  similar  to  one 
presented  in  Chapter  4 for  the  Phase  II  data  has  been  repeated  for  the  Trial  Two 
data.  Figure  2!)  shows  the  rating  of  the  route  selection  model  as  a function  of 
the  perception  model  used,  for  two  different  classes  of  decision  makers 
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Figure  29.  — Overall  Model  Rating  (Trial  Two) 
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(those  preferring  Model  One  alternatives  over  other  alternatives,  those  not 
preferring  Model  One  alternatives)  as  well  as  for  all  decision  makers.  The 
effects  of  all  other  factors  have  been  averaged. 

The  reader  will  recall  that  during  Phase  n and  Trial  Two  the  subjects 
associated  the  following  qualitative  statements  with  the  ratings : 

100:  equivalent  to  decision  maker's  route  plan, 

95-99:  decision  maker's  plan  preferred,  but  no 
reason  can  be  given  for  preference , 

70-94:  decision  maker's  plan  slightly  preferred, 
with  reason,  and 

0-69:  decision  maker's  plan  strongly  preferred 
with  reason. 

Thus,  among  subjects  preferring  Model  One,  the  decision  maker's  plan  was 
preferred  over  Model  One  route  alternatives , but  no  reason  could  be  cited 
for  this  preference.  On  the  other  hand,  route  alternatives  produced  by  Model 
One  were  rejected  strongly  by  those  subjects  preferring  one  of  the  other  models. 
In  fact,  this  class  of  decision  maker  rejected  all  route  selection  model  results 
with  a score  either  bordering  on  or  falling  within  the  strong  rejection  region. 

Comparing  these  results  with  those  presented  in  Figure  19  of  Chapter 
4 it  is  seen  that  an  overall  improvement  in  route  selection  model  performance 
was  achieved  during  Trial  Two.  Furthermore,  while  the  subjects  continued  to 
strongly  reject  the  route  alternatives  produced  by  the  Acid  Test  Model  and 
Model  Three,  the  class  of  subjects  preferring  Model  One  was  unable  to  cite 
any  reason  for  rejecting  route  plans  prepared  by  Model  One.  In  Phase  II  these 
subjects  slightly  preferred  their  own  plans,  with  reason.  Thus,  it  is  seen 
once  more  that  if  one  were  forced  to  construct  a route  selection  model  with 
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only  one  percieved  threat  model,  Model  One  would  be  the  best  choice.  It  has 
achieved  a level  of  acceptability  among  subjects  preferring  Model  One  alter- 
natives over  other  alternatives  that  is  considered  satisfactory.  In  fact,  as 


discussed  in  the  next  section,  it  may  be  difficult  to  ever  achieve  higher  results 
with  any  model.  However,  to  choose  the  one  perception  model  to  the  exclusion 
of  all  others  would  be  a foolhardy  venture.  A class  of  decision  makers  exists 
that  would  strongly  reject  the  results  produced  by  this  model.  In  fact,  Figure 
29  reveals  that  it  may  be  necessary  either  to  revise  the  entire  route  selection 
model  or  to  develop  a new  perception  model  in  order  to  achieve  satisfactory 
acceptance  of  route  alternatives  among  these  subjects. 

Quantitative  Measures.  The  reader  will  recall  that  the  Trial  Two  subjects 
were  provided  a set  of  quantitative  descriptors  for  each  route  alternative  they 
were  to  evaluate.  It  was  hoped  that  these  descriptors  would  permit  the  subjects 
to  perform  a more  objective  comparison  between  their  own  route  plans  and 
those  prepared  by  the  model.  Unfortunately,  it  is  not  obvious  that  such  a 
result  was  achieved  in  Trial  Two. 

Figures  30  through  34  are  scatter  diagrams  showing  the  relationships 
between  ratings  given  the  various  route  alternatives  and  five  quantitative 
descriptors  associated  with  the  alternatives.  The  data  from  which  these 
diagrams  were  prepared  appears  as  Table  F.5  in  Appendix  F.  As  may  be 
seen,  there  is  no  discernible  correlation  between  alternative  rating  and  any 
of  the  quantitative  descriptors.  Furthermore,  it  is  apparent  that  the  subjects 
were  hardly  concerned  with  these  factors  when  they  were  constructing  their 
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Figure  32.  — Rating  as  a Function  of  Time  Exposed  to  Supporting 
Weapon  Only 
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Figure  34.  — Rating  as  a Function  of  Time  Exposed  to  One  or 
More  Weapons 
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Considering  the  above  results  one  of  two  conclusions  may  be  drawn. 
First,  it  is  possible  that  none  of  the  five  factors  considered  in  Figures  30 
through  34  are  important  to  the  subjects  during  route  selection.  However, 
this  is  unlikely  since  all  five  factors  are  related  to  exposure,  and  all  subjects 
have  indicated  minimization  of  exposure  as  a principle  of  choice  in  route 
selection. 

The  other  conclusion  that  might  be  drawn  is  that  the  subjects , in  vio- 
lation of  the  instructions  they  were  given  did  not  use  the  factors  to  assist  in 
rating  the  route  alternatives.  Unfortunately,  there  is  no  way  to  determine 
whether  or  not  this  actually  was  the  case.  However,  it  is  a likely  conclusion. 

The  net  result  of  the  above  observations  is  that  it  is  unknown  whether 
or  not  the  Trial  Two  subjects  experienced  difficulty  in  discerning  true  physical 
differences  between  route  alternatives.  Furthermore,  even  if  such  difficulties 
did  exist,  it  is  unknown  whether  the  quantitative  descriptors  were  helpful  in 
alleviating  them.  Nevertheless,  it  would  appear  from  the  data  in  Figures  30 
through  34  that  the  difficulties  did  exist,  but  the  subjects  still  failed  to  use  the 
descriptors. 

With  such  apparently  arbitrary  results,  it  is  considered  very  fortunate 
that  the  subjects  rated  the  performance  of  the  route  selection  model  as  high 
as  they  did.  In  fact,  it  would  appear  that  any  rating  of  95  or  above  (decision 
maker's  route  plan  preferred,  but  for  no  reason)  should  be  about  the  best  over- 
all performance  that  can  be  achieved.  Thus,  the  rating  received  by  Model  One 
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from  subjects  preferring  Model  One  alternatives  over  other  alternatives  (see 


preceding  section)  is  considered  satisfactory.  Recommendations  for  future 
research  along  these  lines  will  be  presented  in  Chapter  7. 

Predicting  Mode  of  Fire.  The  last  analysis  required  in  this  chapter  is  a test 
of  the  procedure  proposed  earlier  for  predicting  the  mode  of  fire  that  a decision 
maker  will  select  in  a given  attack  situation.  The  reader  will  recall  that  the 
procedure  is  as  follows: 

For  a particular  route  selection  situation,  and  using  one 
of  the  three  threat  perception  models,  determine  a route 
that  includes  a hover  fire  attack.  Denote  the  overall 
threat  of  the  route  as  T^p.  Repeat  the  analysis  for  a 
running  fire  route  to  obtain  Tpp.  If  THp  < T^p  then 
predict  that  the  decision  maker  will  prefer  the  hover  fire 
result.  Otherwise,  predict  that  the  decision  maker  will 
prefer  the  running  fire  route  result. 

In  preparing  for  the  Trial  Two  Experiment,  values  of  T^and  T ^ 
were  determined  for  every  combination  of  subject,  route  and  threat  perception 
model  considered.  These  data  appear  in  Appendix  F at  Table  F.6.  The  pro- 
posed procedure  was  then  applied  to  predict  the  route  alternative  that  each 
subject  would  prefer  for  each  combination  of  situation  and  perception  model. 

The  results  of  this  exercise  appear  in  Table  10  under  fhe  heading  of  Predicted 
Firing  Mode  Preference.  For  example,  it  is  predicted  that  for  route  situation 
one  subject  six  will  prefer  the  hover  fire  alternative  over  the  running  fire 
alternative  if  the  threat  perception  model  used  is  the  Acid  Test  Model. 
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Table  10 


In  examining  this  class  of  data  in  Table  10  it  is  seen  that  one  of  the 
difficulties  discussed  at  the  time  the  procedure  was  developed  is  a common 
occurrence.  That  is,  different  predictions  are  made  depending  upon  which  of 
the  three  threat  perception  models  is  under  consideration.  Out  of  twelve  com- 
binations of  subject  and  situation,  the  three  predictions  are  in  agreement  only 
four  times.  Therefore,  it  is  apparent  as  suggested  earlier  that  the  procedure 
can  be  applied  with  validity  only  after  the  threat  perception  model  which  applies 
to  a given  decision  maker  has  been  determined.  Of  course,  at  this  time  such 
information  is  not  available.  All  that  is  known  is  that  most  of  the  subjects 
apparently  preferred  the  overall  route  results  produced  by  Model  One  during 
Trial  Two.  Thus,  one  might  use  the  predicted  firing  mode  preferences  pre- 
pared for  Model  One  routes  in  testing  the  validity  of  the  procedure  for  those 
subjects  who  preferred  Model  One  alternatives.  The  results  of  such  an  approach 
will  be  discussed  in  a paragraph  below. 

Another  class  of  data  appearing  in  Table  10  is  the  actual  firing  mode 
preference  as  determined  from  rating  data  appearing  in  Table  F.4  of  Appendix 
F.  For  example,  in  route  situation  one,  subject  six  rated  the  running  fire 
alternative  prepared  with  the  Acid  Test  Model  higher  than  he  rated  the  hover 
fire  alternative.  Thus,  it  is  assumed  that  he  preferred  the  running  fire  alter- 
native. In  those  cases  where  the  same  rating  was  given  to  both  alternatives, 
it  is  assumed  the  routes  were  equally  desirable. 
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In  examining  this  class  of  data  it  is  seen  that  there  is  little  indication  of 
a uniform  firing  mode  preference  across  all  three  of  the  threat  perception 
models.  That  is,  in  only  three  cases  (excluding  ties)  out  of  twelve  does  a 
subject  pick  the  same  mode  of  fire  from  the  routes  produced  by  all  three  of 
the  threat  perception  models.  This  may  be  a further  indication  that  the  subjects 
were  not  particularly  biased  against  the  procedures  used  to  represent  either 
type  of  attack  during  Trial  Two.  However,  in  comparing  the  actual  firing  mode 
preferences  with  the  predicted  firing  mode  preferences,  it  is  seen  that  the 
prediction  procedure  apparently  failed  to  achieve  a very  high  level  of  accuracy. 
For  example,  if  ties  are  excluded,  Acid  Test  preferences  are  predicted  cor- 
rectly in  only  three  cases  out  of  ten,  Model  One  preferences  are  predicted 
correctly  in  three  cases  out  of  seven,  and  Model  Three  preferences  are 
predicted  in  six  cases  out  of  eleven.  Even  if  ties  are  counted  as  one-half, 
the  accuracy  figures  are  4 out  of  12,  5-1/2  out  of  12,  and  6-1/2  out  of  12. 
Finally,  if  the  analysis  is  restricted  to  Model  One  routes  and  subject  12  is 
excluded  because  he  did  not  prefer  Model  One  routes  overall,  the  accuracy 
figure  counting  ties  is  only  7 out  of  10.  While  this  is  a higher  accuracy  per- 
centage than  others  obtained  above,  it  still  does  not  represent  a very  success- 
ful performance. 

A final  class  of  data  in  Table  10  is  the  firing  mode  that  the  subjects 
actually  selected  for  their  own  routes  during  the  Phase  II  Experiment.  In 
comparing  these  data  with  the  predicted  preference  data,  it  is  seen  that  the 
prediction  procedure  is  no  more  accurate  in  predicting  subject  route  firing 
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modes  than  it  is  in  predicting  firing  mode  preferences.  The  procedure  is 
correct  only  about  fifty  percent  of  the  time.  ^ 

The  data  outlined  above  are  insufficient  to  classify  the  prediction  pro- 
cedure as  being  invalid.  As  stated  previously,  the  procedure  is  not  intended  to 
predict  a firing  mode  preference  nor  the  actual  choice  of  a firing  mode  with 
certainty.  Rather,  it  is  recognized  that  the  intrapersonal  variation  of  a 
decision  maker  will  cause  him  to  give  different  rankings  to  route  selection 
model  results  in  repeated  trials  of  the  same  decision  problem.  It  will  also 
cause  him  to  possibly  select  different  modes  of  fire  to  be  employed.  Trial 
Two  represents  only  one  realization  of  such  a problem.  Therefore,  it  is 
impossible  at  this  time  to  say  with  certainty  that  the  firing  mode  prediction 
procedure  is  or  is  not  valid.  While  the  results  outlined  above  are  disappointing 
they  do  not  form  a justification  for  rejecting  the  procedure.  Instead,  further 
experimentation  is  required  to  obtain  more  statistical  evidence. 
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CHAPTER  6 

FORT  ORD  EXPERIMENT 
by  Dave  Bitters 

Introduction 

Based  upon  the  results  of  the  experiments  conducted  at  Fort  Rucker,  an 
additional  experiment  was  conducted  at  Fort  Ord  for  the  primary  purpose  of 
using  experienced  pilots  that  were  also  trained  in  the  mid-intensity  combat 
tactics.  A secondary  purpose  was  to  "Iter  the  procedure  to  permit  more  ob- 
jectivity in  comparing  model  and  subject  routes . 

Procedural  Modification 

The  procedural  modification  was  motivated  by  the  Fort  Rucker  subject, 
preferences  for  their  own  routes  even  when  calculations  by  the  route  selection 
model  showed  that  the  subject  routes  were  more  exposed.  Of  course,  the  pre- 
vious statement  may  be  very  biased  toward  the  route  selection  models  because 
their  exposure  measures  may  be  unlike  those  used  by  the  subjects  in  actually 
selecting  routes.  Nevertheless,  the  procedure  used  at  Fort  Rucker  needs  to  be 
modified  to  reduce  the  bias  produced  by  explicitly  using  the  subject  plots  as  the 
standard  for  comparison  with  the  route  selection  model  plots. 

The  procedural  modification  was  to  present  all  routes  for  comparison 
without  any  specific  identification  as  to  which  routes  were  subject  routes.  The 
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intent  was  to  deemphasize  the  subject  routes  and  to  disguise  the  subject  route 


so  that  they  may  not  even  be  recognized  by  the  subject. 

Subject  Qualifications 

The  United  States  Army  has  an  ample  supply  of  qualified  helicopter 
pilots  that  have  combat  experience;  however,  most  of  them  have  only  experi- 
enced combat  in  Southeast  Asia.  Accordingly,  most  are  only  experienced  in 
what  would  be  considered  low-intensity  combat.  The  helicopter  pilot  in  Vietnam 
has  not  had  to  cope  with  anti-aircraft  weapons  (such  as  Redeye)  on  a large  scale. 
It  is  understood  that  most  helicopter  pilots,  however  well  qualified  they  may 
otherwise  be,  are  net  experienced  in  the  tactics  and  doctrine  of  the  attack  heli- 
copter in  the  mid-intensity  environment.  Indeed,  such  tactics  and  doctrine  had 
been  formulated  only  a short  time  before  this  research  was  performed  in  the 
Basic  Attack  Helicopter  Team  (BAHT)  experiments  (43.5  and  43.6)  at  Fort  Ord, 
California. 

One  might  argue  that  the  model  validation  results  obtained  from  the  Ohio 
National  Guard  and  Fort  Rucker  experiments,  as  presented  in  earlier  chapters, 
are  of  somewhat  limited  value  precisely  because  the  subjects  used  in  those  ex- 
periments ware  not  trained  in  the  mid-intensity  doctrine.  For  this  reason,  a 
sample  of  opinions  from  pilots  trained  in  mid-intensity  warfare  is  included  in 
this  report.  The  assumption  was  that  although  the  pilots  so  sampled  may  not 
have  ever  faced  an  antiaircraft  weapon  as  potent  as  the  Redeye,  at  least  they 
would  be  intellectually  familiar  with  the  problems  posed  by  a significant 
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^ antiaircraft  threat,  and  that  their  behavior  (and  accordingly  their  responses  to 

the  situations  presented  to  them)  would  be  modified  accordingly. 

As  of  early  May,  1971,  the  only  unit  known  to  be  operationally  trained 
in  mid-intensity  warfare  doctrine  was  the  155th  Helicopter  Company  (Attack), 
stationed  at  Fort  Ord,  California  and  attached  to  the  USACDC  Experimentation 
Command  (USACDCEC).  The  155th  had  conducted  extensive  maneuvers  against 
hard-point  targets  under  conditions  of  a simulated  significant  antiaircraft  threat, 
in  connection  with  the  BAHT  experiments  mentioned  earlier.  They  were,  there- 
fore, familiar  with  procedures  such  as  knap  of  the  earth  flying,  running  and 
hover  fire,  and  antiaircraft  evasive  tactics.  They  were  also  well  informed  of 
the  capabilities  and  characteristics  of  the  14.5mm,  23mm  and  REDEYE  antiair- 
craft weapons.  In  short,  the  155th  Helicopter  Company  was  an  ideal  unit  from 
which  to  draw  subjects  for  a final  comparison  experiment. 

A two-phase  experiment  was  arranged  at  Fort  Ord,  California  for  May 
18,  1972,  and  June  23,  1972.  The  experiment  involved  ten  volunteer  subjects 
from  the  155th  Helicopter  Company.  Each  subject  filled  out  a volunteer  infor- 
mation form  from  which  the  information  appearing  in  Tables  11  through  14  was 
obtained.  The  information  in  these  tables  indicates  that  the  subjects  were  gen- 
erally highly  experienced  with  their  aircraft,  were  seasoned  combat  veterans 
in  terms  of  total  combat  flight  time,  and  were  generally  familiar  with  the  prob- 
lems of  mid-intensity  combat.  Notice  in  particular  that  all  subjects  had  been 
engaged  by  enemy  antiaircraft  fire  (Table  13),  that  all  but  one  had  engaged  hard- 
point  targets  (Table  14),  and  that  all  but  one  had  received  formal  instruction  in 
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Table  11 


Distribu-  4 Cpt 

tion  2 lLt 


Table  14 


I 


Yes  2.75  HEAT 


Table  15 


23mm  8640  2000  REDEYE  B770  4440  REDEYE  8770  4440  23mm  8640  2000 
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mid-intensity  warfare  tactics . Judging  from  the  extensive  flight  experience  and 
the  number  of  months  at  Fort  Ord  (Table  11)  of  the  subject  who  did  not  claim 
formal  mid-intensity  training,  it  seems  reasonable  to  assume  that  he  was 
thoroughly  familiar  with  the  doctrine  through  on-the-job  training.  In  short, 
the  subjects  interviewed  appeared  to  be  ideally  qualified  to  assist  in  the  valida- 
tion of  the  DYNFLITE route  selection  model.  Moreover,  their  cheerful  coopera- 
tion made  working  with  them  a pleasure. 

Phase  I Experimentation 

The  experimentation  session  conducted  on  May  18,  1972,  followed  the 
procedure  outlined  in  Chapter  3 with  one  exception,  to  be  discussed  below. 
Otherwise,  any  deviation  in  the  experimental  procedure,  with  whatever  bias 
might  have  resulted  therefrom,  is  totally  due  to  the  fact  that  the  experiments  at 
Fort  Rucker  and  Fort  Ord  were  conducted  by  different  individuals.  Every  effort 
was  made  to  guarantee  that  the  format  of  Chapter  3 was  carefully  followed. 

The  only  planned  deviation  from  the  format  of  the  Fort  Rucker  experi- 
ment dealt  with  the  map  boards  given  to  four  of  the  subjects.  The  reader  may 
recall  that  the  second  part  of  the  Phase  I experiment  was  a map  board  exercise 
in  which  each  subject  was  given  one  of  four  map  boards.  Each  map  board  con- 
sisted of  a computerized  representation  of  a contour  map  of  the  battlefield. 

Each  map  had  a designated  starting  point,  and  two  sites  indicated  as  locations 
of  enemy  weapons.  Each  subject  was  given  two  overlays:  one  showing  foresta- 
tion and  the  other  indicating  regions  of  intervisibility  with  respect  to  each  of  the 
two  weapon  sites.  Thus  the  subjects  were  given  more  information  than  they 
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might  have  in  an  actual  tactical  situation.  In  an  actual  situation  a pilot  probably 
would  not  have  intervisibility  information  at  hand.  He  would  have  the  benefit 
of  contour  maps  and  aerial  photographs  showing  forested  areas,  however.  Thus 
it  was  natural  to  wonder  if  the  absence  of  line -of -sight  information  would  sig- 
nificantly alter  the  preference  routings  made  by  the  subjects  during  the  Phase  n 
experiment.  Accordingly,  four  subjects  were  denied  access  to  the  intervisibility 
overlays.  The  subjects  were  instructed  to  draw  two  routes,  one  to  each  target, 
on  plastic  overlays  provided  to  them  (see  Chapter  4).  The  routes  were  supposed 
to  reflect  the  flight  plans  that  the  subject  would  select  were  he  actually  confronted 
with  the  situations  represented.  Each  subject  had  access  to  three  computer- 
generated isometric  plots  of  the  battlefield  showing  the  forested  areas  in  green. 

The  subjects  were  thoroughly  briefed  on  the  type  of  aircraft  to  be  used 
for  the  hypothetical  attack,  as  well  as  on  the  type  of  armament  they  would  have. 
They  were  given  range  and  lethality  characteristics  of  the  three  types  of  weapons 
that  they  might  encounter.  They  were  also  given  kill  probability  data  for  their 
own  weapons.  Examples  of  the  instructions  given  to  the  subjects  can  be  found 
in  Appendix  B. 

The  design  format  used  in  the  map  board  experiment  is  shown  in  Table  15, 
which  lists  the  subjects  by  number  and  indicates  which  map  board  each  was  given. 
The  "prime"  notation  after  the  map  boards  of  subjects  12,  13,  14,  and  17  indi- 
cates that  those  particular  map  boards  did  not  have  the  intervisibility  overlays 
affixed.  The  rows  of  Table  15  are  to  be  interpreted  as  follows:  Subject  6*  used 

♦Subject  numbers  started  with  6 to  facilitate  comparison  with  the  experi- 
ment at  Fort  Rucker. 
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map  board  A,  which  required  that  his  first  route  be  drawn  to  attack  the  14.5mm 
weapon  at  map  co-ordinates  X = 2630,  Y = 1010.  A second  weapon  (23mm)  at 
co-ordinates  X = 3600,  Y - 1220  posed  a potential  threat  when  in  range  and 
intervisible  with  the  pilot.  For  the  second  route  the  pilot  was  required  to  attack 
the  23mm  with  the  14.5mm  weapon  posing  the  additional  threat.  The  other  lines 
of  Table  15  are  to  be  interpreted  similarly. 

Each  subject  was  asked  to  specify  whether  his  designated  mode  of  attack 
was  to  be  running  fire  or  hover.  The  results  are  shown  in  Table  16. 


Table  16 

Subject  Fire  Mode  Choices 


Subject 

Route  1 

Route  2 

6 

RF 

HF 

7 

RF 

RF 

8 

HF 

HF 

9 

HF 

HF 

10 

HF 

HF 

11 

HF 

HF 

12 

HF 

HF 

13 

RF 

HF 

14 

HF 

HF 

Judging  from  the  written  comments,  one  would  conclude  that  the  choice 
of  mode  of  fire  was  based  largely  on  the  tactical  situation  (e.g. , utilization  of 
existing  cover)  rather  than  on  any  uniformly  applied  doctrine  or  on  convenience 
to  the  pilot.  The  reader  should  therefore  be  cautioned  against  using  the  results 
of  Table  16  as  a basis  for  a regorous  statistical  inference  regarding  the  relative 
merits  of  running  fire  and  hover  fire. 


229 


Phase  II  Experimentation 

The  first  part  of  the  Phase  I experiment  was  designed  to  elicit  sufficient 
information  from  the  subject  to  permit  the  construction  of  his  perceived  threat 
function  (see  Chapter  3).  The  perceived  threat  function  in  turn  is  an  integral 
part  of  the  Acid  Test  decision  model  which  was  discussed  in  Chapter  3.  The 
reader  will  recall  that  decision  models  one  and  three  and  the  Acid  Test  Model 
present  three  rather  divergent  views  of  the  manner  in  which  a pilot  might  approach 
a target,  based  upon  the  way  he  subjectively  assesses  the  threat.  Models  one 
and  three  make  some  rather  bold  assumptions  about  the  way  a pilot  would  react 
in  threatening  and  non-threatening  situations.  The  Acid  Test  Model  on  the  other 
hand  is  more  personalistic  in  the  sense  that  it  does  not  presuppose  a given  be- 
havioral pattern  but  rather  infers  a behavioral  pattern  from  the  subject's  re- 


sponses. The  Acid  Test  Model  in  that  sense  serves  as  a counterpoint  to  the 
other  models,  a control,  so  to  speak. 

The  data  obtained  from  the  first  part  of  the  Phase  I experiment  were  en- 
coded and  incorporated  in  programs  RTSELH  and  RTVAL.  Program  RTSELH 
generates  a route  from  the  starting  point  to  the  target  and  back  to  the  starting 
point,  based  on  the  decision  rule  employed.  That  is,  RTSELH  selects  that  path 
from  the  starting  point  to  the  target  such  that  perceived  threat  is  minimized. 
The  output  of  RTSELH  becomes  the  input  to  RTVAL,  which  evaluates  the  "tacti- 
cal difficulty"  of  each  route  relative  to  the  decision  rule  used,  and  which  gener- 
ates plotter  instructions  permitting  each  route  to  be  traced  on  a vellum  overlay. 
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As  one  changes  the  decision  model,  one  obtains  different  definitions  of 
perceived  threat,  and  hence  one  is  likely  to  obtain  different  routes  for  the  same 
situation.  Indeed,  the  purpose  of  the  Phase  II  experiment  was  to  obtain  an 
a posteriori  comparative  evaluation  by  the  subjects  of  the  routes  generated  by 
the  various  decision  rules. 

The  total  number  of  plots  generated  by  RTSELH/RTVAL  for  each  subject 
was  twelve.  Each  subject  was  given  plots  of  route  1 - running  fire,  route  2 - 
running  fire,  route  1 - hover  fire,  and  route  2 - hover  f.re  for  decision  model  1, 
decision  model  3,  and  the  Acid  Test  Model.  The  starting  co-ordinates  and 
target  locations  in  each  case  corresponded  to  those  which  the  subjects  used 
during  the  map  board  exercise  of  the  Phase  I experiment.  Thus  each  subject 
was  given  model  1,  model  3,  and  Acid  Test  routes  in  both  running  and  hover  fire 
attack  modes,  for  each  of  the  two  situations  (routes  1 and  2)  encountered  in  the 
Phase  I experiment. 

It  was  observed  earlier  that  the  Acid  Test  routes  could  be  viewed  as  con- 
trols for  the  purpose  of  multiple  comparison,  because  they  presumably  repre- 
sent the  subjects'  true  feelings  about  perceived  threat.  However,  better  controls 
do  exist;  namely,  the  routes  which  the  subjects  drew  during  the  Phase  1 map  board 
exercise.  It  is  assumed  that  those  routes  represent  the  a priori  (or  unquantified) 
optimum  routes;  that  is,  it  is  assumed  that  those  routes  would  actually  be  used 
by  the  subjects  in  the  absence  of  evidence  other  than  that  which  was  provided  to 
them  during  the  Phase  I experiment.  Thus  the  subject-generated  routes  could  be 
used  as  a basis  for  comparison  as  was  done  in  the  Port  Rucker  experiment. 
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However,  an  unforeseen  bias  occurred  in  the  Fort  Rucker  experiment  as  a re- 
sult of  the  fact  that  the  subjects  could  identify  their  own  routes  and  were  required 
to  use  them  as  comparison  standards.  Basically  the  difficulty  was  that  the  sub- 
ject routes  were  used  simultaneously  as  comparison  standards  and  as  routes  to 
be  ranked  against  the  RTSELH/RTUAL  routes.  It  turned  out  that  the  dual  roles 
of  the  subject  routes  were  incompatible:  the  subjects  tended  to  systematically 
prefer  their  own  routes  to  those  of  the  three  decision  models.  It  was  believed 
that  the  subject-generated  routes  would  not  necessarily  be  uniformly  preferred 
if  their  identity  could  be  somehow  concealed.  Accordingly,  the  subject-routes 
of  the  map  board  exercise  were  encoded  and  run  through  program  PVAL.  Pro- 
gram PVAL  produced,  as  outt  ut,  instructions  to  the  plotter  which  allowed  the 
subject-routes  to  be  plotted  on  vellum  in  identically  the  same  foremat  as  those 
of  RTSELH/RTUAL.  Thus  it  was  possible  to  externally  mask  the  identity  of 
the  subject  routes. 

The  rules  of  the  Phase  II  experiment  were  altered  slightly  from  those  of 
the  Fort  Rucker  experiment  to  accommodate  the  absence  of  a prespecified  stan- 
dard. Each  subject  was  given  four  groups  of  plots  designated  route  1 - running 
fire,  route  2 - running  fire,  route  1 - hover  fire,  and  route  2 - hover  fire. 

Each  group  contained  at  least  three  but  not  more  than  four  plots.  Those  groups 
containing  four  plots  corresponded  to  the  route  and  choice  of  firing  mode  made  by 
the  subject  during  the  map  board  exercise  of  Phase  I.  Thus,  for  example,  if  a 
subject  chose  hover  fire  for  route  1 and  running  fire  for  route  2 in  drawing  his 
plots,  those  groups  designated  route  1 - hover  fire  and  route  2 - running  fire 
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would  contain  only  three  plots — Model  1,  Model  3,  and  Acid  Test.  The  plots 


were  arranged  randomly  within  each  group. 

Thus,  in  all  but  two  cases  the  subjects  received  a total  of  fourteen  plots. 
Subjects  10  and  14  received  only  twelve  plots . Regrettably  an  anomaly  in  the 
starting  co-ordinates  given  to  subjects  10  and  14  in  the  map  board  exercise  re- 
sulted in  routes  which  were  believed  to  be  significantly  different  than  those  which 
would  have  resulted  had  the  starting  co-ordinates  been  the  same  as  those  input 
to  RTSELH.  The  error  unfortunately  was  not  discovered  until  it  was  too  late 
for  a correction.  The  absence  of  the  subject-plots  for  subjects  10  and  14  con- 
stitutes a potential  bias  as  will  be  described  later  in  this  chapter.  The  resulting 
bias  may  tend  to  make  certain  evidence  of  questionable  value.  A full  treatment 
of  the  problem  will  be  given  in  the  section  on  the  analysis  of  results.  Regard- 
less, it  was  deemed  expedient  to  omit  the  subject-plots  for  subjects  10  and  14 
and  proceed  with  the  experiment  as  originally  planned. 

The  subjects  were  instructed  to  make  a multiple  rank  comparison  within 
each  group.  That  is,  the  plots  were  to  be  ranked  by  the  subjects,  first  qualita- 
tively, then  quantitatively.  The  route  deemed  best  by  the  subject  was  to  be 
given  a value  of  100,  with  the  other  routes  being  given  lesser  numerical  scores 
according  to  the  scheme  outlined  in  Chapter  4.  Tie  scores  were  permitted  in 
the  cases  where  the  subject  was  indifferent  between  two  routes.  (Note,  in  this 
context,  that  in  some  cases  Models  1 and  3 generated  identical  routes.  Indiffer- 
ence is  axiomatic  in  such  situations.)  A typical  ranking  for  the  two  route-1 
groups  is  shown  in  Table  17.  The  symbols  in  the  table  are  Rt  1 for  Route  1, 
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RF  for  Running  Fire,  HF  for  Hover  Fire,  S for  Subject  Plot,  AT  for  Acid  Test 


■ 


Model,  Ml  for  Model  One,  and  M3  for  Model  Three 

The  subjects  were  then  instructed  to  rank  adl  route-1  plots  together  with 
all  route-2  plots  according  to  the  rules  described  above.  Thus  in  essence  the 
categories  route  1 - running  fire  and  route  1 - hover  fire  were  combined,  and 
the  combined  category  was  ranked;  the  route  2 combined  category  was  ranked 
in  the  same  manner.  Thus,  a typical  aggregated  ranking  is  shown  in  Table  18. 


Table  17 

Typical  Grouped  Route  Ranking 


Plot 

Value 

Rt  1 - RF  - S 

100 

Rt  1 - RF  - AT 

90 

Rt  1 - RF  - M2 

85 

Rt  1 - RF  - Ml 

50 

Rt  1 - HF  - AT 

100 

Rt  1 - HF  - M3 

98 

Rt  1 - HF  - Ml 

25 

Table  18 

Typical  Aggregated  Route  Ranking 


Plot 

Value 

Rt  1 - HF  - AT 

100 

Rt  1 - RF  - S 

99 

Rt  1 - HF  - M3 

95 

Rt  1 - RF  - AT 

80 

Rt  1 - RF  - M3 

75 

Rt  1 - HF  - Ml 

25 

Rt  1 - RF  - Ml 

1 

For  the  purpose  of  the  analysis  the  grouped  rankings  were  ignored.  The 
main  reason  for  making  the  subjects  do  the  grouped  rankings  was  to  familiarize 
them  with  the  plots.  A secondary  reason  was  to  familiarize  them  with  the  rank- 
ing scheme. 

The  rules  for  the  Phase  II  experiment  were  essentially  the  same  as  those 
for  the  map  board  experiment.  (Of  course  in  the  Phase  II  experiment  the  subjects 
had  to  evaluate  routes  rather  than  construct  them.)  All  subjects  had  access  to 
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the  three  computer  drawn  isometric  maps  of  the  battlefield.  Subjects  6-11  had 
the  line-of -sight  overlays  while  subjects  12,  13,  14  and  17  did  not.  Each  subject 
was  given  a copy  of  the  descriptive  material  which  accompanied  the  map  board 
in  the  Phase  I experiment.  Use  of  the  material  was  for  the  purpose  of  refamil- 
iarizing the  subject  with  the  nature  of  the  problem  (recall  that  Phases  I and  II 
were  conducted  more  than  a month  apart).  The  entire  Phase  II  experiment  was 
conducted  in  about  one  hour . 

During  the  course  of  the  ranking  exercise,  references  to  the  decision 
models  were  deliberately  kept  as  obscure  as  possible.  The  subjects  were  told 
only  that  the  routes  were  derived  from  the  information  supplied  during  Phase  I. 
Naturally,  some  of  the  subjects  wondered  if  reproductions  of  their  routes  drawn 
during  the  map  board  exercise  were  among  the  plots  given  them.  The  response 
to  their  query  was  merely  that  their  routes  had  been  used  in  the  analysis.  It  is 
unlikely  that  many  of  the  subjects  were  fooled,  especially  in  view  of  the  fact  that 
some  of  the  groups  given  them  contained  four  plots  and  others  contained  three. 
Thus  the  danger  of  biased  results  was  not  completely  overcome.  The  subjects 
were  urged  however  to  do  nothing  more  than  compare  the  plots  given  to  them, 
without  regard  to  the  map  board  exercise  of  the  previous  experiment.  There  is 
no  reason  to  believe  that  the  subjects  did  not  make  a sincere  effort  to  comply 
with  the  instructions . 

When  the  ranking  exercise  was  completed,  the  subjects  were  instructed 
to  identify  the  routes  which  they  believed  most  closely  resembled  the  ones  they 
had  constructed  during  the  map  board  exercise  of  Phase  I. 
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Analysis  of  the  Results 


The  first  question  to  be  answered  is  whether  or  not  the  masking  of  the 
subject  plots  was  successful.  One  can  be  absolutely  certain  that  the  rankings 
of  the  plots  are  unbiased  (in  the  sense  that  they  are  influenced  by  the  prior  pref- 
erence of  the  subject -generated  plots)  only  if  the  subjects  are  not  able  to  easily 
distinguish  their  own  routes  from  among  the  others.  The  above  question  can  be 
admirably  addressed  by  a straightforward  inferential  technique.  Eight  of  the 
ten  subjects  were  given  packets  containing  fourteen  plots,  two  of  which  were 
computerized  representations  of  the  routes  they  had  themselves  constructed  a 
month  earlier.  (Recall  that  a programming  anomaly  rendered  the  subject  plots 
for  subjects  10  and  14  useless.)  The  subjects  were  instructed  to  select  from 
among  the  fourteen  plots  the  two  which  they  believed  most  closely  represented 
their  own  routes.  Recall  that  the  fourteen  plots  were  divided  into  four  groups, 
two  of  which  contained  four  plots  each.  Clearly  the  subjects  would  not  even  con- 
sider the  other  groups;  prior  information  told  them  those  groups  contained  no 
subject-plots.  Thus  the  subjects  were  to  select  one  plot  from  each  of  two  groups 


of  four  given  to  them.  There  are 


a-  ra- 


16  distinguishable  pairs  of  plots 


which  can  be  selected  in  this  fashion.  The  number  of  pairs  which  contain  no 
/ 3\  /3\ 

subject  plot  is  ( ^ lx(  J = 9.  Thus  the  number  of  pairs  which  contain  at  least 
one  correctly  identified  plot  is  16  - 9 = 7.  Hence  if  a subject  were  merely 
guessing  which  two  plots  were  representations  of  his  own  routes,  the  probability 
of  his  making  at  least  one  correct  selection  would  be  . With  eight  subjects 
making  selections  of  plots,  the  hypothesis  that  the  subjects  cannot  necessarily 
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identify  their  own  routes  and  therefore  make  random  selections  would  imply 
the  following:  the  probabilities  that  exactly  u of  the  subjects  correctly  identify 
at  least  one  plot  is  given  by  the  binomial  distribution.  That  is, 

P(u)=(u)(jj)  (it)  u = 0,  1,  2, ... 8 , 

where  there  are  eight  subjects. 

Only  one  subject  failed  to  identify  at  least  one  of  his  own  plots . The 
probability  that  seven  or  more  subjects  make  a correct  identification  is 

r<7>  * p<8,  - 8 (1-)  (£)  — .0151 . 

More  significantly,  only  two  subjects  failed  to  identify  both  plots  correctly.  The 
probability  of  this  event  is 


Thus  the  evidence  overwhelmingly  suggests  that  the  subjects  were  not  guessing; 
that  is,  after  more  than  a month  they  were  generally  able  to  identify  correctly 
the  plots  which  depicted  their  own  routes.  The  effort  to  mask  the  subject-plots 
in  order  to  eliminate  the  bias  discussed  earlier  was  therefore  clearly  a failure. 
Thus  the  question  of  bias  in  favor  of  the  subject-plots  remains  open  throughout 
the  present  discussion. 
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There  are  two  possible  reasons  for  the  failure  of  the  masking  attempt. 
First,  of  course,  it  is  possible  that  randomly  selected  subjects  would  generally 
be  able  to  identify  their  own  plots  under  any  circumstances.  It  would  thus  be 
futile  to  attempt  to  design  an  experiment  which  would  eliminate  subject  bias. 
Second,  it  is  possible  that  the  subjects  were  able  to  deduce  at  the  beginning  of 
the  Phase  II  experiment  that  two  of  the  routes  were  their  own.  Aside  from  the 
fact  that  such  a strategy  appears  to  be  an  obvious  one  to  use,  two  major  clues 
could  have  aided  the  subjects  in  their  deduction:  1)  two  of  the  subjects  had  only 
twelve  plots;  2)  those  subjects  who  received  fourteen  plots  got  two  groups  of 
three  plots  and  two  groups  of  four  plots . The  lack  of  uniformity  both  in  the 
number  of  plots  given  to  the  subjects  and  in  the  number  of  plots  within  each 
group  appear  in  retrospect  to  provide  rather  obvious  clues.  Of  course  the 
subjects  were  still  required  to  make  the  appropriate  selections,  but  the  possible 
presence  of  prior  clues  as  to  the  intent  of  the  experimenter  might  have  made 
their  task  much  easier.  It  is  indeed  unfortunate  if  the  efforts  to  mask  the  iden- 
tity of  the  subject  plots  were  negated  by  the  lack  of  uniformity  described  above. 
About  the  only  alternative  would  be  to  produce  some  bogus  plots  so  that  each 
subject  would  receive  four  groups  of  four  each. 

Regardless  of  the  potential  bias  described  above,  there  is  no  reason  to 
believe  that  the  subjects  made  less  than  an  honest  attempt  to  evaluate  the  rela- 
tive merits  of  the  plots  presented  to  them,  according  to  the  instructions  and  in 
the  spirit  of  the  experiment.  Thus,  although  the  significance  of  the  results  may 
be  somewhat  suspect,  they  should  not  be  dismissed  as  altogether  meaningless. 
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Because  the  subject-plots  were  not  identified  as  standards  for  compari- 
son, the  analysis  approach  used  in  the  Fort  Rucker  experiment  was  not  applicable 
to  the  Fort  Ord  data.  Thus  it  is  necessary  to  explain  briefly  how  the  results 
were  compiled  and  what,  if  any,  significance  they  possess. 

Two  separate  analyses  were  in  fact  performed.  It  was  believed  that  they 
would  produce  somewhat  different  results  and  thus  would  provide  a basis  for 
comparison  by  the  decision  maker.  Although  the  data  compilation  technique  was 
similar  in  each  case,  it  will  be  helpful  to  describe  each  separately. 

Unweighted  Analysis.  Table  19  presents  typical  responses  provided  by 
the  subjects  and,  as  such,  will  provide  the  basis  of  a tutorial  illustration  of  the 
manner  in  which  the  unweighted  rankings  were  obtained. 


Table  19 

Typical  Subjects'  Responses 


Plot 

Subject  A 
Values 

Subject  B 
Values 

Rt  2-HF-S 

100 

75 

Rt  2-HF-AT 

99 

0 

Rt  2-HF-M1 

25 

10 

Rt  2-HF-M3 

25 

100 

Without  regard  to  the  numerical  values  given  by  the  subjects,  we  can  obtain  the 
rankings  shown  in  Table  20. 
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Table  20 
Subject  Rankings 


Subject  A 

Rt  2-HF-S 
Rt  2-HF-AT 

Rt  2-HF-M1 , Rt  2-HF-M3 


Subject  B 

Rt  2-HF-M3 
Rt  2-HF-S 
Rt  2-HF-M1 
Rt  2-HF-AT 


Thus,  preference  is  indicated  by  relative  vertical  position:  subject  A 
prefers  plot  S to  plot  AT,  plot  AT  to  plots  Ml  and  M3,  and  declares  Ml  and  M3 
to  be  tied,  for  instance.  It  is  evident  from  Table  20  that  subjects  A and  B by  no 
means  agree  on  the  ranking.  Therefore,  the  rankings  must  be  combined  in  some 
sort  of  consistent  fashion.  The  approach  which  was  taken  was  to  determine  what 
amounts  to  an  average  ranking . Thus  the  rankings  in  Table  20  need  to  be  en- 
coded in  a consistent  numerical  format.  To  do  this  one  can  develop  a transitive* 
ranking  function  which  can  be  represented  in  matrix  format: 


S 

AT 

Ml 

M3 


Subject  A 
AT  Ml 


1 2 

0 1 

-1  0 

-1  0 


♦The  transitive  property  implies  that  if  X is  preferred  to  Y and  Y is  pre- 
ferred to  Z,  then  X is  preferred  to  Z. 
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The  interpretation  of  the  matrix  entries  is  the  following:  M(P,  Q)=K  means 
thaf  olot  P is  preferred  to  plot  Q by  K units.  There  are  three  cases.  First,  as 


in  the  case  of  M(AT,S)  = -1  the  quantity  is  negative.  This  means  that  in  fact  S 
is  preferred  to  AT  by  one  unit.  (Note  that  in  fact  M(S,AT)  = 1).  Second,  as 
illustrated  by  M(M1,M3)  = 0,  if  the  quantity  is  zero,  then  Ml  and  M3  are  tied 
in  the  preference  ranking.  The  third  use  of  course  is  where  M(P, Q)>0,  as  illus- 
trated by  M(S,M3)  = 2.  Thus,  for  example,  S is  preferred  to  M3  by  two  units. 
One  can  make  a similar  representation  of  subject  B's  rankings. 


S 


Subject  B 


AT  Ml 


M3 


2 

0 

1 

3 


Two  important  facts  should  be  noted:  1)  for  a given  transitive  ranking 
matrix,  any  row  will  produce  the  same  rank  ordering  (this  is  because  it  is 
always  true  that  M(P,Q)  = -M(Q,P));  2)  in  order  to  recover  the  original  ranking 
from  a transitive  ranking  matrix  it  is  sufficient  to  know  the  algebraic  sign  of 
each  entry  (thus,  for  instance  P is  preferred  to  Q if  and  only  if  M(P,Q)>0 ). 

To  obtain  the  composite  (or  average)  ranking,  one  simply  adds  the  two 
matrices  componentwise  (or  averages  the  components).  It  is  then  possible  to 


obtain  the  composite'  ranking: 


241 


Subject  (A,B) 

S AT  Ml  M2 


One  can  thus  recover  the  composite  ranking  as  shown  in  Table  21  for  our  example. 


Table  21 

Illustrative  Composite  Ranking 


Subject  (A , B)  Ranking 

Rt  2-HF-S 

(because  M(S,AT),  M(S,M1), 

M(S,M3)>  0) 

Rt  2-HF-M3 

(because  M(M3,AT),  M(M2,M1)>0 

Rt  2-HF-AT, 

Rt  2-HF-M1 

(because  M(AT,M1)  = 0) 

Weighted  Analysis.  The  analysis  utilizing  the  actual  numerical  values 
provided  by  the  subjects  required  an  initial  value  judgment  by  the  analyst.  Con- 
sider, for  example,  the  response  of  hypothetical  subject  B (Table  19).  Recall 


that  the  weighted  ranking  was: 

Rt  2-HF-M3  - 100 

Rt  2-HF-S  - 75 

Rt  2-HF-M1  - 10 

Rt  2-HF-HT  - 0 
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Thus  subject  B preferred  plot  M3  to  all  others.  Presumably  the  degree 
of  preference  is  indicated  by  the  accompanying  numbers.  The  numbers  could 
easily  be  transformed  into  percentages.  Thus  one  might  interpret  the  above 
results  to  mean  that  plot  S is  75%  as  good  as  plot  M3.  The  problem  which 
arises  is  in  comparing  plots  Ml  and  AT,  for  instance,  or  plots  S and  Ml. 
Suppose  that  plot  M3  were  eliminated  from  the  comparison.  Would  the  subject 
simply  add  25  to  the  other  three  numbers  to  obtain  the  ranking: 


Rt  2 - HF-S 

100 

Rt  2-HF-M1 

35 

Rt  2-HF-AT 

25 

or  would  he  employ  the  scaling  factor 

100 

75  ? In  the  latter  case  the  ranking  would 

be: 

Rt  2-HF-S 

100 

RT  2-HF-M1 

13I 

1J3 

Rt  3-HF-AT 

0 

Again,  if  plot  S were  removed  one  could  obtain  either  of  the  following  rankings 
(by  adding  90  in  the  first  case  or  multiplying  by  10  in  the  second): 


Rt  2-HF-M1 

100 

Rt  2-HF-M1 

100 

Rt  2-HF-HT 

_ 

90 

Rt  2-HF-AT 

_ 

0 

Of  course  the  responses  which  the  subject  would  actually  give  under  the  modified 

situations  described  above  can  only  be  conjectured.  However,  the  presence  of 
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a response  of  zero  for  plot  AT  indicates  total  rejection;  one  might  reasonably 

assume  from  the  nature  of  the  response  that  the  subject  would  award  AT  a score 

of  zero  regardless  of  the  scores  assigned  to  any  of  the  other  routes.  That  is,  in 

a pairwise  comparison  of  AT  with  any  other  route,  in  which  the  other  route  were 

mandatorily  scored  100,  AT  would  be  given  a score  of  zero.  The  scaling  factor 

method  of  comparison  is  consistent  with  this  assumption. 

One  can  define  a ranking  function  M(P,Q)  as  follows:  Suppose  the  number 

associated  with  plot  P (subject's  response)  is  a;  suppose  also  that  the  subject 

a 

assigns  number  b to  plot  Q.  Then  define  M(P,Q)  = ^ . Note  that  P is  preferred 

to  Q if  and  only  if  M(P,Q)  > 1.  From  a technical  point  of  view  the  reader  may 

a 

wonder  what  the  significance  of  M(P,Q)  = — is  in  the  case  where  b = 0.  Of 
course  theoretically  there  should  be  no  objection  to  having  M(P,Q)  = oo  . That 
would  simply  mean  that  Q is  never  preferred  (the  case  ^ was  not  encountered 
so  it  need  not  be  considered).  In  view  of  the  practical  limits  of  the  computer, 
however,  it  was  deemed  expedient  to  replace  a response  of  0 by  .001  wherever 
encountered.  If  interpreted  carefully  the  analysis  will  yield  the  same  conclusions 
regardless.  For  the  hypothetical  example  of  Table  19  the  resulting  weighted 


matrices  are: 


Subject  A 


s 

AT 

Ml 

M3 

1 

100 

100 

100 

99 

25 

25 

99 

1 

99 

99 

100 

25 

25 

25 

25 

100 

99 

1 

1 

25 

25 

i 

1 

im 

W 

J. 

Subject  B 

S 

AT 

Ml 

M3 

1 

75 

75 

75 

.001 

10 

100 

.001 

1 

.001 

.001 

75 

10 

100 

10 

10 

1 

10 

75 

.001 

100 

100 

100 

100 

1 

75 

.001 

10 

The  combined  ranking  is  obtained  in  this  case  by  multiplying  the  entries  of  the 
matrices  together  component  by  component/  Thus  if  M^(P,Q)  and  M^(P,Q) 


mm 


represent  the  ranking  function  values  for  subjects  A and  B,  the  corresponding 


value  of  the  combined  ranking  function  is  M (P,Q)  = M (P,Q)M  (P,Q). 

(A , B)  A B 

Accordingly  the  composite  ranking  function  for  the  above  example  is 


S 

AT 

Ml 

M2 


S 

1 

1.3  x 10" 

3.3  x 10' 
3.3  x 10 


Subject  (A , B) 

AT  Ml 


7.6  x 104 
1 

2.5  x 103 
2.5  x 104 


30 

4.0  x 10"4 
1 

10 


M3 

3 

4.0  x 10"5 


which  yields 


S 

AT 

Ml 

M3 

s 

I ■ 

00 

30 

3 ^ 

AT 

1 . 

1 

0 

0 

Ml 

3.3  x 10"2 

00 

1 

10-1 

M2 

3.3  x 10"1 

00 

10 

1 

upon  replacing  .001  by  zero  in  the  subject  B matrix.  One  can  then  recover  the 
ranking  by  invoking  the  rule  that  P is  preferred  to  Q if  and  only  if  M(P,Q)  > 1. 
Thus  one  obtains  the  ranking 
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Table  22 

Illustrative  Ranking  from  Weighted  Analysis 


The  reader  is  invited  to  compare  Tables  21  and  22. 

Factors  Analyzed 

The  experimental  design  was  utilized  to  develop  a variety  of  rank  order- 
ings to  consider  a variety  of  situations.  Recall  (Table  15)  that  four  different 
route  selections  were  made  by  each  subject,  and  four  of  the  subjects  were  denied 
line  of  sight  information. 

The  first  ranking  which  was  developed  could  be  termed  the  overall  rank- 
ing. In  the  overall  ranking,  all  subject  rankings  except  those  of  subjects  10  and 
14  were  combined  according  to  the  procedure  outlined  above.  Other  rankings 
were  made  to  compare  various  effects: 

1)  Route.  The  route  1 rankings  of  subjects  6,  7,  8,  9,  11,  12,  13,  and 
17  were  combine  d and  compared  with  the  route  2 rankings  of  the  same  subjects 
to  see  if  the  factor  of  the  particular  route  involved  made  any  difference.  It  should 
be  noted  that  one  would  not  a priori  suspect  a significant  difference  because 


Subject  (A , B) 
Rt  2 - HF  - S 
Rt  2 - HF  - M3 
Rt  2 - HF  - Ml 
Rt  2 - HF  - AT 


I 
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abstractly  one  route  should  not  pose  more  difficulties  than  any  other  (that  is, 
one  would  expect  the  results  to  be  the  same  even  if  all  of  the  routes  numbered 

1 were  renumbered  2 and  vice  versa). 

2)  Line  of  sight.  The  routes  of  subjects  6,  7,  and  11  were  compared 
with  those  of  subjects  12,  13,  and  17.  Subjects  6 and  12  had  map  board  A, 
subjects  7 and  13  had  B,  and  subjects  11  and  17  had  D.  However,  subjects 
6,7,  and  11  had  line  of  sight  information  whereas  12,  13,  and  17  did  not.  Thus 
a comparison  of  similar  situations  with  and  without  intervisibilities  information 

was  effected. 

3)  Target  weapon.  Notice  from  Table  15  that  the  following  routes  in- 
volved attacking  a 14.5mm  target:  6R1,  7R1,  8R1,  9R1,  12R1,  13R1  (routes 
10R1  and  14R1  were  omitted  from  this  analysis  since  the  subject  plots  were 
missing  from  the  rankings  of  subjects  10  and  14).  On  the  other  hand,  routes 
6R2,  7R2,  8R2,  9R2,  11R1,  12R2,  13R3,  and  17R1  attacked  a 23mm  target. 
Thus,  the  two  categories  just  described  were  tabulated  and  compared  to  deter- 
mine if  the  factor  of  the  specific  weapon  attacked  made  a significant  difference. 

4)  Map  board  (retabulated).  The  entire  procedure  for  both  the  weighted 
and  unweighted  individual  ranking  functions  was  repeated,  deliberately  omitting 
the  subject-plots  in  each  case  (thus  for  each  subject  two  groups  of  six  items 
each  were  ranked:  e.g.,  Rt.  1-RF-AT,  Rt  1-RF-M1,  Rt  1-RF-M3,  Rt  1-H1- 
AT,  Rt  l-HF-Ml,  Rt  1-HF-M3).  The  reason  for  doing  so  was  to  be  able  to 
take  advantage  of  the  data  provided  by  subjects  10  and  14.  The  reader  will  note 
from  Table  15  that  subjects  6 and  12  had  map  board  A;  7,  9,  and  13  had  B;  8,  10, 
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and  14  had  C;  and  11  and  17  had  D.  Thus  those  four  categories  were  tabulated 
and  compared  to  see  if  the  effect  of  the  map  board  was  significant. 

5)  Line  of  sight  (retabulated).  Computations  similar  to  those  in  (2) 
above  were  made  with  the  retabulated  rankings.  At  a sacrifice  of  omitting  the 
subject  plots  from  this  ranking,  two  further  rankings  could  be  included  in  the 
analysis.  Thus  the  routes  of  subjects  6,7,  10,  and  11  were  compared  with 
those  of  subjects  12,  13,  14,  and  17. 

It  should  be  noted  that  each  of  the  analyses  performed  above  was  done 
according  to  both  the  weighted  and  unweighted  formats.  A summary  of  the 
factorial  analysis  schemes  is  presented  in  Table  23. 

Results 

Table  24  presents  the  numerical  scores  and  ranking  awarded  to  each 
route  by  each  subject.  The  numerical  scores  were  provided  by  the  subjects  and 
the  rankings  were  of  course  derived  from  the  scores.  The  reader  should  bear 
in  mind  that  the  rankings  presented  below  constitute  the  basis  of  the  analyses 
to  follow. 

The  results  of  the  various  analyses  can  now  be  stated.  The  overall 
rankings  obtained  by  combining  the  rankings  of  all  but  subjects  10  and  14  are 
shown  in  Tables  25  and  26. 
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Table  23 


Factorial  Analysis  Schemes 


Comparison 

Nature 

Routes 

Description 

1 

Route 

Rte.  1,  Subj.  6,7,8, 
9,11,12,13,17  vs. 
Rte.  2,  Subj.  6,7,8, 
9,11,12,13,17 

Compares  overall 
effect  of  route  1 to 
overall  effect  of 
route  2 

2 

Line  of  sight 

Subj . 6,7,11  (Rtes . 

1 & 2)  vs. 

Subj.  12,13,17  (Rtes. 
1 & 2) 

Compares  subjects 
with  line  of  sight 
information  to  those 
without 

3 

Target  weapon 

Subj.  6,7,8,9,12,13 
(Rte.  1)  vs. 

Subj.  6,7,8,9,12,13 
(Rte.  2),  11,17  (Rte. 
1) 

Compares  effect  of 
attacking  14.5mm 
with  effect  of  attack- 
ing 23  mm 

4 

Map  board  (re- 
tabulated) 

Subj.  6,12  (Rtes.  1 & 
2)  vs. 

Subj.  7,9,13  (Rtes. 

1 & 2)  vs. 

Subj.  8,10,14  (Rtes. 

1 & 2)  vs. 

Subj.  11,17  (Rtes.  1 
& 2) 

Compares  effects 
of  the  four  map 
boards 

5 

Line  of  sight 

Subj.  6,7,10,11  (Rtes. 
1 & 2)  vs. 

Subj.  12,13,14,17 
(Rtes . 1 & 2) 

Same  as  (2)  above 
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Table  25 

Unweighted  Ranking 


(*: 

4. 

5. 

6. 


Subject 

Hover  fire  acid  test 
Hover  fire  model  3 ' 

Running  fire  acid  testy 
Running  fire  model  3 
Hover  fire  model  1 
Running  fire  model  1 


Table  26 

Weighted  Ranking 


Ranking 


Value 


1. 

Subject 

1 1 

2. 

Hover  fire  acid  test 

2.102  x 105 

3. 

Running  fire  acid  test 

2.360  x 105 

4. 

Running  fire  model  3 

5.518  x 105 

5. 

Hover  fire  model  1 

3.935  x 10b 

6. 

Hover  fire  model  3 

6,582  x 109 

7. 

Running  fire  model  1 

1.092  x 10i£> 

The  reader  will  recall  that  numerical  scores  were  obtained  for  the 
weighted  analysis  by  multiplying  corresponding  entries  of  the  various  ranking 
matrices  involved.  For  the  purpose  of  having  a uniform  comparison  standard, 
the  subject  plot  was  arbitrarily  selected.  Thus  for  example  if  one  were  given 
the  ranking  matrix 
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S 

AT 

Ml 

M3 

s 

/ 1 

1.01 

4 

4 

AT 

1 .99 

1 

3.99 

3.99 

Ml 

1 .25 

.23 

1 

1 

M3 

\ .25 

.23 

1 

1 

one  would  deduce  the  ranking 

1.  S 

2.  AT 

/ 3.  Ml  \ 

\ 3.  M3  / 


by  inspecting  the  values  in  any  row.  Thus,  for  example  one  could  associate  the 
numbers  in  the  row  marked  S (or  any  other,  for  that  matter)  with  the  deduced 
ranking.  The  associated  ranking  would  be  the  same  in  any  case.  Using  row  S 
one  would  have : 


Rank 

Value 

1.  S 

1 

2.  AT 

1.01 

/ 3.  Ml  \ 

4 

\3.  M3  / 

4 

The  values  thus  presented  would  represent  the  ranking  of  each  of  the  routes 
relative  to  the  subject  route. 

The  convention  to  be  followed  throughout  this  report  will  be  the  following: 
in  the  case  of  the  weighted  ranking  tabulations,  if  the  subject  route  enters  the 
analysis,  the  numerical  values  will  always  be  those  which  are  obtained  relative 
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to  the  subject  route.  In  the  cases  in  which  the  subject  route  is  omitted,  the 
ranking  values  will  be  relative  to  running  fire  acid  test.  The  reader  will  un- 
doubtedly recognize  that  the  choices  are  quite  arbitrary.  However,  at  least 
in  the  cases  in  which  a subject  route  exists  for  comparison  they  are  not  totally 
without  foundation,  i.e. , the  assumption  is  reasonable  that  the  subject  would 
prefer  to  use  his  own  route  for  a uniform  paired  comparison  standard. 

A few  words  should  be  said  about  the  overall  ranking  results.  Notice 
that  the  unweighted  (ordinal)  and  weighted  rankings  are  quite  similar.  In  fact 
the  only  difference  is  that  hover  fire  model  3 moves  from  position  3 (tied  with 
running  fire  acid  test)  to  position  6.  A plausible  explanation  of  this  change  of 
location  is  that  hover  fire,  model  3,  was  generally  fairly  highly  ranked  by  the 
subjects,  but  that  two  or  three  of  them  found  that  it  produced  a highly  impractical 
route.  A glance  at  Table  24  would  tend  to  confirm  the  above  remarks.  Notice 
also  that  the  subject  route  was  uniformly  preferred  to  the  others. 

The  relative  ranking  values  are  also  rather  revealing.  The  reader  will 

recall  that  a slight  modification  of  the  rankings  provided  by  the  subjects  was 

made:  namely,  a subject  score  of  zero  for  a route  was  arbitrarily  changed  to 

10-3.  Thus  if  a subject  gave  route  S a score  of  100,  and  route  RF-M1  a score 

-3  5 

of  zero,  the  associated  relative  ranking  would  be  M(S,RF-M1)  = 100/10  = 10  . 

It  is  quite  unlikely  that  products  of  the  order  of  magnitude  of  105  would  result 
unless  at  least  one  subject  scored  the  second  of  the  paired  routes  zero.  Thus 
inspection  of  Table  26  suggests  that  at  least  one  subject  regarded  each  route  as 
absolutely  valueless.  A glance  at  Table  24  indeed  confirms  this  suspicion.  A 
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A more  detailed  look  at  the  results  will  be  provided  by  the  factor  analyses  con- 
tained in  Table  27  through  31. 

Tables  26  through  31  clearly  show  that  the  preference  rankings  are  far 
from  unanimous.  In  fact,  in  looking  at  Tables  27  to  31  one  might  wonder  if  any 
discernible  patterns  of  consensus  are  present,  or  if  the  rankings  therein  pre- 
sented reveal  only  incoherent  variation  (noise).  Clearly  some  sort  of  statistical 
methodology  is  required  to  test  hypotheses  regarding  adequacy  of  ranking.  The 
one  to  be  employed  here  is  the  pairwise  application  of  a sign  test  discussed  on 
pp.  16-8  to  16-9  of  National  Bureau  of  Standards  Handbook  #91,  Experimental 
Statistics  by  M.  G.  Natrella.  Briefly,  the  test  is  a two-sided  one  to  determine 
if  two  products  (processes,  routes,  etc.)  differ  in  average  performance.  Suppose, 
for  example,  that  two  routes  designated  A and  B were  given  the  following  numeri- 
cal scores  by  the  various  subjects: 


Subject 

A 

B 

A/B 

6 

99 

87 

+ 

7 

45 

72 

- 

8 

83 

24 

+ 

9 

96 

96 

X 

10 

36 

40 

- 

11 

72 

60 

+ 

12 

59 

35 

+ 

13 

92 

19 

+ 

14 

81 

73 

+ 

17 

52 

27 

+ 

The  entries  in  the  column  designated  A/B  are  obtained  by  comparing  the  values 
given  in  column  A with  the  respective  values  appearing  in  column  B:  a plus  sign 
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Table  29 


Table  30 


Table  31 

Map  Board  Comparisons , Recomputed:  6,  12  Vs.  7,  9,  13  Vs 


is  entered  if  the  entry  in  column  A exceeds  that  in  column  B.  If  the  entry  in  B 


exceeds  that  in  A,  a minus  sign  is  recorded.  In  case  the  two  entries  are  equal, 
a large  X is  entered  in  column  A/B.  Notice  that  there  are  ten  pairs  of  numbers, 
nine  of  which  produce  either  a plus  or  a minus  in  column  A/B.  Referring  to 
Table  A -33  of  NBS  handbook  91,  one  sees  that  rejection  of  the  hypothesis  that 
A and  B are  not  significantly  different  at  the  . 10  level  requires  that  eight  or 
more  plus  signs  appear  in  column  A/B.  At  the  .25  level  one  would  reject  the 
hypothesis  based  on  the  appearance  of  seven  or  more  plus  signs.  Thus  one 
would  reject  the  hypothesis  at  the  .25  level  but  not  at  the  .10  level. 

All  of  the  pairwise  analyses  to  be  discussed  below  follow  precisely  the 
format  just  outlined.  In  each  case  the  significance  level  was  taken  to  be  . 10. 

Tables  32  and  33  exhibit  the  basic  raw  data  from  which  the  rank  tests 
were  made.  The  entries  in  the  cells  of  Table  32  were  obtained  as  follows. 

There  are  = 21  pairs  of  plots  which  can  be  formed  from  the  basic  seven 
model  and  subject  plots,  i.e.,  RF-AT,  HF-AT,  RF-M1,  HF-M1,  RF-M3, 
HF-M3,  and  the  subject  plots.  Those  pairs  are  represented  across  the  top  of 

g 

Table  32  where  a pair  has  an  upper  plot  and  a lower  plot,  e.g. , the  pair 
would  consist  of  the  subject  plot  as  an  upper  plot  and  the  running  fire  acid  test 
plot  as  the  lower  plot.  In  each  case  the  entry  in  a cell  was  obtained  by  compar- 
ing the  upper  plot  with  the  lower  plots,  using  the  entires  contained  in  Table  24. 

g 

Thus,  for  example  RAT  (rte.  1 - subj.  6)  is  a comparison  of  the  subject  plot 
vs.  the  running  fire  acid  test  plot  for  subject  6,  route  1.  Referring  to  Table 
24,  one  sees  that  the  subject  plot  in  that  case  was  given  a rating  of  99,  whereas 
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the  running  fire  and  test  plot  was  given  a score  of  80.  Since  99-80  is  greater 

g 

than  zero,  an  entry  of  plus  was  recorded  in  the  (rte.  1 - subj.  6)  cell. 

A similar  exercise  was  performed  to  obtain  Table  33.  However,  in  Table  33, 
composite  rankings  were  encoded,  rather  than  individual  rankings.  The  com- 
posite rankings  which  were  utilized  were  the  weighted  rankings  in  each  case. 

First,  statistical  analyses  were  performed  to  determine  the  signifi- 
cance of  the  components  within  the  various  rankings.  For  instance,  the  over- 
all ranking  (Table  26)  combines  sixteen  component  rankings,  namely  rte.  1 - 
subj.  6,  7,  8,  9,  11,  12,  13,  17;  rte.  2 - subj.  6,  7,  8,  9,  11,  12,  13,  17. 

Thus  for  each  pair  of  routes  there  are  sixteen  pairs  of  numbers  which  result 

in  a column  of  sixteen  plus,  minus,  or  X symbols.  The  columns  of  symbols 
were  analyzed  to  determine  significance  (or  lack  thereof).  The  results  of  the 
analysis  are  specified  below. 

Overall  ranking.  The  subject  plot  was  significantly  preferred  to  each 
of  running  fire  route  1,  hover  fire  route  1.  The  running  fire  acid  test  route 
was  significantly  preferred  to  running  fire  route  1;  likewise  hover  fire  route  1 
was  significantly  preferred  to  running  fire  route  1.  In  all  other  cases,  either  a 
significant  difference  was  not  detected  or  there  were  insufficient  samples  to 
make  a determination.  Pictorially,  the  statistical  analysis  of  the  overall 

ranking  can  be  represented  as  a partial  order:  S 

"f  RF  -AT 

HF-M1  / 

* X 

RF-M1 

Route  1 ranking.  RF-AT  was  significantly  preferred  to  RF-M1.  No 


other  significance  was  observed. 
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Route  3 rankings.  S was  significantly  preferred  to  RF-M1  and  HF-M1. 
Line  of  sight  - subjects  6,  7.  11.  RF-M3  was  significantly  preferred  to 
RF-M1. 


Line  of  sight  - subjects  12,  13,  17.  RF-AT  was  significantly  preferred 
to  RF-M1,  HF-AT  was  significantly  preferred  to  RF-M1  and  to  HF-M1;  HF-M3 
was  significantly  preferred  to  RF-M1. 

Line  of  sight,  retabulated  ~ subjects  12,  13,  14,  17.  RF-AT  was  signifi- 
cantly preferred  to  RF-M1;  HF-M3  was  significantly  preferred  to  RF-M1. 

Weapon  alternative  14.5mm.  RF-AT  was  significantly  preferred  to 
HF-M1. 

Weapon  alternative  23mm.  S was  significantly  preferred  to  RF-M1  and 
HF-M1;  HF-AT  was  significantly  preferred  to  RF-M1. 

Map  board  A.  Insufficient  data  existed  to  make  a statistical  determina- 


tion. 

Map  board  B.  No  significant  differences  were  detected. 

Map  board  C.  Insufficient  data  existed  to  make  a statistical  determina- 
tion. 

Map  board  D.  No  significant  differences  were  detected. 

The  reader  should  be  cautioned  that  the  statistical  procedure  employed 
to  obtain  the  above  results,  being  distribution  free,  is  not  very  powerful.  Be- 
cause a significant  difference  was  not  detected  in  the  relative  value  of  a pair  of 
routes , it  is  by  no  means  necessarily  the  case  that  a significant  difference 
would  not  emerge  if  enough  subjects  were  polled.  If  the  entire  population  of 
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U.S.  Army  helicopter  pilots  were  sampled,  doubtless  more  definitive  state- 


ments could  be  made.  Lack  of  significance  can  be  attributed  to  one  of  three 
factors:  1)  too  small  a sample,  2)  too  many  ties  (more  than  20%  render  the 
particular  test  employed  useless),  or  3)  genuine  failure  of  the  test  to  reveal 
significance.  For  a significant  difference  to  be  detected,  there  must  be  many 
more  plus  signs  than  minus  signs  (or  vice  versa).  In  many  cases,  due  to  the 
small  sample  sizes,  significance  could  not  be  asserted  without  unanimous  agree- 
ment, for  instance.  One  should  regard  the  cases  wherein  significance  was  not 
detected  as  open  problems.  That  is,  from  a philosophical  point  of  view  one 
can  make  no  definitive  statement  based  on  the  small  samples  taken  as  to  whether 
there  would  be  a true  consensus  among  the  pilot  population  as  a whole  with  re- 
spect to  a given  pair  of  routes,  or  not. 

If  one  accepts  the  rankings  presented  in  Tables  26  through  31  as  being 
reasonable  measures  of  consensus,  one  can  perform  a somewhat  more  inter- 
esting analysis.  Table  33,  as  mentioned  earlier,  was  obtained  by  encoding  the 
preference  rankings  of  the  weighted  analyses  presented  in  Tables  26  through  31. 
Of  course,  some  of  the  information  will  be  redundant  due  to  the  fact  that  each 
of  the  rankings  in  Tables  26  through  31  was  obtained  by  combining  a given  sub- 
set of  the  set  of  subject  responses  for  routes  1 and  2.  Nevertheless,  according 
to  table  A -33  of  NBS  Handbook  91.  for  paired  samples  of  size  7,  seven  signs  of 
the  same  type  constitutes  evidence  for  a significant  difference  in  the  pair  of 
plots  in  question,  at  both  the  .05  and  . 10  significance  level.  That  is,  unanimity 
constitutes  a statistical  consensus  at  the  .05  and  .10  level.  Likewise,  for  13 


I 

I 

I 

I 


I 
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paired  samples  ten  or  more  outcomes  of  the  same  type  constitutes  a statisti- 
cally significant  consensus . 

A glance  at  Table  33  reveals  that  of  the  first  six  columns,  five  exhibit 
the  conditions  sufficient  for  consensus:  S/RFAT,  S/RF-M1,  S/RF-M3,  S/HF- 
Ml,  S/HF-M3.  Since  these  columns  all  contain  plus  signs  one  obtains  the  in- 
formation that  the  subject  plot  is  significantly  preferred  to  RF-AT,  RF-M1, 
RF-M3,  HF-M1  and  HF-M3.  With  respect  to  the  remaining  columns,  RF-AT/ 
RF-M1,  RF-M3/HF-M3,  and  HF-HT/HF-M3  exhibit  the  sufficient  property  for 
statistical  significance  in  terms  of  plus  signs.  In  terms  of  minus  signs  RF-Ml/ 
RF-M3,  RF-M1/HF-AT,  R F-M1/HF-M1 , RF-M1/HF-M3,  and  RF-M3/HF-AT 
exhibit  statistical  significance.  Thus  one  can  conclude  that  RF-AT,  RF-M3, 
HF-AT,  HF-M1,  and  HF-M3  are  all  preferred  to  RF-Ml;  that  HF-AT  is  pre- 
ferred to  RF-M3;  and  that  RF-M3  and  HF-AT  are  both  preferred  to  HF-M3. 

The  above  results  can  be  represented  diagrammatically  by  the  following  partial 
order: 


No  definitive  statements  can  be  made  about  relationships  between  the  remaining 
pairs. 


The  reader  is  urged  to  refer  to  Table  26.  Note  that  the  above  partial 
order  is  consistent  with  the  strict  linear  order  of  Table  26  in  the  following 
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is  a chain  in  both  the  above  partial  order  and  in  Table  26.  The  fact  of  con- 
sistency of  partial  orderings  may  not  in  itself  be  particularly  surprising.  How- 
ever, the  fact  that  such  a large  number  of  the  pairings  (over  half)  exhibit  com- 
patible results,  statistically  speaking,  is  highly  interesting. 

The  foregoing  analyses  constitute  a sort  of  distribution-free  analysis 
of  variance.  In  that  context  one  discerns  the  following  pattern:  within  each 
factor  there  is  considerable  individual  variation  (few  statistically  significant 
paired  differences  were  observed);  however,  the  average  rankings  of  the  factors 
manifested  a remarkable  consistency,  as  seen  by  the  partial  ordering  shown 
above. 

In  conclusion,  some  general  qualitative  observations  should  be  made. 

One  of  the  interesting  problems  considered  in  the  overall  experiment  was  whether 
the  absence  of  line-of-sight  information  made  any  difference  in  the  subject  re- 
sponses. There  is  some  evidence  that  it  did.  Notice  (Table  24)  that  subject  17 
ascribed  a rank  of  zero  to  all  plots  except  his  own  and  the  running  fire  acid  test 
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plot.  Subject  17  was  required  to  attack  a target  known  to  be  a REDEYE  in  his 
route  2 plot.  Pictorially,  the  situation  with  which  he  was  confronted  was 


1 


analogous  to  Figure  35: 

X / covered 
target  ^ 

7 A 

target-subject  intervisibility  corridor 
Figure  35 

That  is,  it  was  possible  for  the  subject  to  pass  very  close  to  the  target  without 
being  intervisible.  Every  decision  model  except  running  fire  acid  test  took 
advantage  of  that  fact  to  bring  the  firer  near  the  target  via  a totally  covered 
route.  Now  of  course  subject  17  was  not  able  to  recognize  that  his  approach 
would  always  be  covered,  since  he  was  purposely  denied  the  line  of  sight  over- 
lay. He,  therefore,  reasoned  that  each  decision  model  except  RF-AT  would 
bring  him  dangerously  close  to  an  extremely  lethal  situation  in  an  unacceptable 
fashion  (namely,  on  a seemingly  uncovered  route).  Thus  he  understandably 
exhibited  absolutely  no  enthusiasm  for  the  decision  models  except  RF-AT. 

In  light  of  the  response  of  subject  17,  it  might  be  well  in  future  experi- 
mentation with  the  decision  models  to  provide  each  subject  with  line-of-sight 
information.  In  a real  combat  situation  the  pilot  would  have  an  opportunity  to 
study  the  actual  terrain  over  which  he  would  have  to  fly,  and  presumably  the 
results  of  his  study  would  be  tantamount  to  having  a line-of-sight  overlay. 
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The  results  of  chapter  five  of  this  report  indicate  that  the  pilots  at 


Fort  Rucker  tended  to  favor  decision  model  1.  The  Fort  Rucker  results  are 
clearly  at  variance  with  those  presented  in  this  chapter,  wherein  running  fire  - 
model  1,  for  instance,  is  almost  uniformly  ranked  below  the  rest.  There  is, 
however,  a plausible  explanation.  The  purpose  in  conducting  the  Fort  Ord  ex- 
periment in  the  first  place  was  to  obtain  a sample  of  pilots  trained  in  mid- 
intensity warfare  tactics.  Now  the  reader  will  recall  (Chapter  2)  that  decision 
Model  1 was  described  as  modeling  an  extremely  conservative  behavior.  De- 
cision Model  1 places  a uniformly  high  penalty  on  being  exposed  to  a hostile 
weapon,  regardless  of  the  range.  Decision  Model  3,  on  the  other  hand,  is  less 
conservative;  the  exposure  penalty  is  a function  of  the  range  as  well  as  whether 
the  pilot  is  covered  or  uncovered.  The  Acid  Test  Model  is  supposed  to  capture 
the  flavor  of  the  subject's  individual  subjective  reaction  to  danger  as  a function 
of  range,  azimuth,  exposure  time,  etc.  Generally  it  is  viewed  as  less  conser- 
vative than  either  decision  Model  1 or  decision  Model  3. 

It  is  altogether  likely  that  the  difference  in  the  results  of  the  two  experi- 
ments can  be  explained  in  terms  of  an  experience  factor.  The  Fort  Ord  pilots 
were  used  to  mid-intensity  antiaircraft  evasion  tactics,  were  familiar  with 
the  more  lethal  weapons , and  would  therefore  logically  be  willing  to  accept  a 
greater  risk  than  would  pilots  not  trained  in  mid -intensity  warfare.  Several 
subjects  at  Fort  Rucker  indicated  verbally  that  the  conservative  decision  model 
was  more  appealing  to  them  in  terms  of  results  produced , even  though  it  seem- 
ingly contradicted  their  own  perceived  threat  functions  (see  Chapters  3 and  5). 
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the  Fort  Ord  subjects  which  confirm  the  conjecture  that  decision  Model  1,  in 
particular,  was  found  to  be  too  conservative.  The  reason  for  the  preference 


of  the  Acid  Test  Model  remains  open  to  speculation.  The  best  one  can  do  is 
assert  that  there  did  turn  out  to  be  a correlation  between  the  experience  of  the 


subjects  with  respect  to  mid-intensity  warfare  training  and  conservativeness 
of  the  decision  model  which  they  preferred,  viz. , the  training  made  the  pilots 


more  aggressive. 


CHAPTER  7 


HELICOPTER  ROUTE  SELECTION  MODEL 
(DYNFLITE) 

by  Gordon  M.  Clark 


Introduction 

This  chapter  presents  an  overall  evaluation  of  the  proposed  route  selec- 
tion models  and  a specification  of  the  model  to  be  used  in  DYNFLITE . Recall 
that  DYNFLITE  is  to  be  used  in  DYNTACS  X for  selecting  helicopter  attack 
routes.  In  addition,  some  observations  are  made  concerning  insights  to  the 
route  selection  decision  process.  Finally,  recommendations  are  made  con- 
cerning future  research  for  DYNFLITE.  More  specifically,  four  questions  are 
addressed  in  this  chapter: 

] . What  is  the  best  route  selection  model? 

2.  Has  an  acceptable  route  selection  model  been  developed? 

3.  What  are  the  DYNFLITE  design  specifications? 

4.  What  future  research  should  be  performed? 


Evaluation  of  Candidate  Model  Performance 


An  Important  Qualification 

Prior  to  evaluating  the  performance  of  the  candidate  route  selection 
models,  an  important  qualification  must  be  made  that  applies  to  all  conclusions 
drawn  throughout  this  chapter.  This  qualification  pertains  to  the  Acid  Test 
Mode'.,  defined  in  Chapter  3,  which  is  an  important  candidate  route  selection 
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model.  In  fact  this  model  is  the  most  interesting  in  the  respect  that  individual 
perceptions  of  threat  in  diverse  situations  are  reflected  in  the  model  as  input 
tactical  doctrine.  That  is,  the  model  attempts  to  quantify  the  threat  perceptions 
of  individual  decision  makers  to  a much  higher  degree  than  the  other  two  models. 

The  qualification  assumes  that  the  Acid  Test  Model  as  implemented  in  the  route 
selection  model  of  Chapter  2 accurately  reflects  decision-maker  preferences 
concerning  exposure  to  enemy  threat.  In  Chapter  3 on  page  112,  a comparison 
was  made  showing  that  decision  makers  did  not  have  additive  disutilities  as 
assumed  in  the  route  selection  model.  Thus,  the  routes  selected  employing 
the  Acid  Test  Model  may  not  be  the  preferred  routes  based  upon  the  decision- 
maker inputs.  In  fact,  the  comparison  in  Chapter  3 indicates  errors  in  threat 
calculations  on  the  order  of  100%  are  possible.  However,  all  conclusions  in 

I 

this  research  assume  that  these  errors  will  not  effect  the  results  since  a re- 

] 

vision  to  the  dynamic  programming  algorithm  was  beyond  the  scope  of  the  re- 
search. 

Preferred  Model 

With  the  above  qualification  in  mind,  the  preferred  route  selection  model 
will  be  identified.  As  discussed  in  Chapter  5,  the  Fort  Rucker  experiments  in- 
dicated that  decision  makers  could  be  divided  into  two  classes  with  respect  to 
their  preferences  for  a route  selection  model,  i.e. , those  preferring  Model  One 

j 

and  those  preferring  some  other  model.  Overall,  the  decision  makers  preferred 
Model  One.  This  indicated  that  most  decision  makers  used  a very  simple  approach 
to  evaluating  routes,  viz. , minimization  of  total  exposure  time  without  regard 
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to  target  distance  or  azimuth.  This  preference  for  Model  One  shows  that  most 
decision  makers  of  the  Fort  Rucker  group  responded  differently  in  evaluating 
complete  routes  than  they  did  in  evaluating  shorter  route  segments. 

On  the  other  hand,  the  Fort  Ord  decision  makers  preferred  the  Acid 
Test  Model  when  a hover-fire  was  employed.  In  fact,  the  ratings  for  the  hover- 
fire  Aied  Test  Model  routes  were  not  significantly  different  than  the  subject 
plots.  This  result  shows  that  the  route  evaluations  by  the  Fort  Ord  decision 
makers  are  consistent  with  their  route  segment  evaluations.  Since  the  Fort  Ord 
decision  makers  were  trained  in  mid-intensity  combat,  these  experiments  indi- 
cate that  the  Acid  Test  Model  is  the  preferred  route  selection  model,  at  least 
when  employing  hover  fire. 

When  considering  running  fire,  the  Acid  Test  Model  is  preferred  over 
Model  One;  however,  the  ratings  for  the  Acid  Test  Model  and  Model  Three  were 
not  shown  to  be  significantly  different.  The  general  conclusion  is  that  the  Acid 
Test  Model  is  preferred  over  the  other  two  models.  Of  course,  confidence  in 
this  conclusion  could  be  increased  with  additional  experimentation. 

Acceptability  of  the  Acid  Test  Model 

Having  chosen  the  Acid  Test  Model,  the  next  step  is  to  estimate  the  de- 
gree of  acceptability  of  the  model.  That  is,  can  the  Acid  Test  Model  take  a 
decision  maker's  inputs  specifying  his  threat  perception  or  tactical  doctrine 
and  determine  acceptable  routes?  The  viewpoint  used  throughout  this  research 
is  that  the  decision  maker  is  the  ultimate  judge  as  to  the  acceptability  of  a 
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route  and  that  the  model  routes  are  compared  against  the  subject  routes  to 
judge  their  acceptability. 

An  acceptable  route  is  defined  as  one  giving  a rating  of  95  or  above  by 
a decision  maker  employing  the  rating  rules  of  Chapter  4.  That  is,  the  model 
route  is  acceptable  when  consciously  compared  against  the  subject  routes  as 
the  standard.  Such  a rating  implies  that  the  decision  maker  can  find  no  defin- 
able fault  in  the  route,  and  at  the  very  worst  would  only  slightly  prefer  his  own 
route  alternative  to  the  one  selected  by  the  model.  It  would  be  desirable  to  re- 
strict the  definition  of  an  acceptable  route  even  further.  However,  it  was  noted 
in  Chapter  5 that  it  may  be  very  difficult  to  ever  achieve  higher  average  ratings 
because  the  population  of  decision  makers  may  be  biased  against  the  model 
routes . 

With  this  standard  in  mind,  the  Fort  Ord  fu  are  analyzed  to  estimate 
the  acceptability  of  the  Acid  Test  Models  routes.  The  Fort  Rucker  results  are 
regarded  as  biased  estimates  since  the  subjects  were  not  trained  in  mid-inten- 
sity tactics.  The  Fort  Ord  Acid  Test  and  Subject  route  ratings  are  shown  in 
Table  34.  Subject  10  data  are  deleted  because  of  the  error  in  the  initial  point 
of  the  subject  routes  noted  in  Chapter  6.  Also,  subject  12,  13,  14,  and  17  data 
are  deleted  because  those  subjects  did  not  have  the  advantage  of  line-of-sight 
overlays.  Table  35  presents  the  averages  for  each  model.  Additional  entries 
are  labeled  as  Maximum-Acid  Test,  and  these  entries  represent  the  maximum 
of  the  running  fire  and  hover  fire  results  in  Table  34  for  each  subject.  The 
motivation  for  taking  the  maximum  of  the  running  fire  and  hover  fire  ratings  is 
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Subject  and  Acid  Test  Model  Ratings 
(Fort  Ord  Experiment) 


Hover  Fire 
Acid  Test 

Running  Fire 
Acid  Test 

Subject 

Subject  6 - Route  1 

100 

80 

99 

- Route  2 

95 

100 

99 

Subject  7 - Route  1 

97 

99 

100 

- Route  2 

97 

0 

75 

Subject  8 - Route  1 

50 

90 

100 

- Route  2 

20 

10 

100 

Subject  9 - Route  1 

40 

40 

100 

- Route  2 

40 

85 

80 

Subject  11  - Route  1 

90 

100 

75 

- Route  2 

90 

80 

75 

Table  35 

Average  Model  Ratings 
Fort  Ord  Data 


Model 

Adjusted  for 
Subiect  as  Standard 

Average 

Score 

Hover  Fire  - Acid  Test 

No 

71.9 

Running  Fire  - Acid  Test 

No 

68.4 

Maximum  - Acid  Test 

No 

82.1 

Subject 

No 

90.3 

Hover  Fire  - Acid  Test 

Yes 

78.8 

Running  Fire  - Acid  Test 

Yes 

75.7 

Maximum  - Acid  Test 

Yes 

90.9 

Subject 

Yes 

100 

to  use  the  best  Acid  Test  Model  result  over  both  firing  tactics.  Table  35 
shows  that  the  subject  route  has  a higher  average  rating  than  the  Acid  Test 
Model  Result.  These  average  ratings  were  adjusted  using  the  methodology  of 
Chapter  6 so  that  the  subject  route  is  regarded  as  the  standard.  That  is,  the 
unadjusted  average  ratings  were  divided  by  .903  so  that  the  average  subject 
route  rating  is  100.  On  this  basis,  the  hover  fire  - acid  test  rating  becomes 

78.8  and  the  running  fire  - acid  test  rating  is  75.7.  Also,  the  adjusted  rating 
for  the  maximum  acid  test  rating  is  90.9.  Referring  to  the  rating  scale  defini- 
tion presented  on  page  119,  these  Acid  Test  Model  values  ranging  from  75.7  to 

90.9  are  in  the  third  category  where  the  subject's  route  is  mildly  preferred 
(on  the  average)  to  the  Acid  Test  Model  route  and  a specific  reason  iv.r  the 
preference  can  be  cited. 

Another  viewpoint  on  the  Acid  Test  Model  acceptability  is  provided  by 
enumerating  the  preferences  shown  in  Table  34  for  each  subject.  On  the  posi- 
tive side,  there  are  six  out  of  ten  cases  where  one  of  the  Acid  Test  Model  routes 
are  preferred  to  the  subject  routes.  On  the  other  hand,  subject  seven  rated  the 
acid  test  - running  fire  route  as  worthless,  i.e. , a score  of  zero,  for  route  2. 

However,  subject  seven  gave  the  acid  test  - hover  fire  route  a rating  of  97 
which  is  better  than  the  75  for  the  subject  route.  The  worst  results  were  ratings 
of  ten  and  twenty  by  subject  eight  on  route  2. 

The  conclusion  from  this  analysis  is  that  the  Acid  Test  Model  can  be  re- 
garded as  acceptable  for  use  in  DYNTACS  X;  however,  the  results  will  not 
always  be  preferred  to  routes  selected  manually.  In  particular,  recognition 
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must  be  given  to  the  low  Acid  Test  Model  ratings  given  by  the  Fort  Rucker  sub- 
jects. On  the  other  hand,  sometimes  the  Acid  Test  Model  routes  will  even  be 
preferred  to  routes  selected  by  a decision  maker  by  his  own  admission.  This 
conclusion  must  be  qualified  by  stating  that  there  is  room  for  improving  the 
model  since  the  average  Acid  Test  Model  ratings  are  below  the  subject  routes 
indicating  a mild  subject  route  preference . 

DYNFLITE  Design  Specifications 

Based  upon  the  above  results , the  following  specifications  are  made 
for  DYNFLITE : 

1.  Use  the  Acid  Test  Model  for  comparing  alternative  routes. 

2.  Constrain  the  model  so  that  hover  fire  can  only  occur  from  a 
covered  position  while  flying  knap -of -the -earth. 

3.  Include  rise  and  fall  time  in  order  to  obtain  intervisibility  with 
target  for  firing  action. 

Future  Research 

When  considering  future  research  in  developing  a helicopter  route  se- 
lection model,  three  problems  with  the  Acid  Test  Model  are  important: 

1.  The  existing  dynamic  programming  algorithm,  presented  in  Chap- 
ter 2,  does  not  find  the  least  threatening  route. 

2.  The  Acid  Test  Model  did  not  rate  high  with  the  Fort  Rucker  subjects. 

3.  Some  of  the  Fort  Ord  subjects  rejected  the  Acid  Test  Model. 
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Ideally,  a route  selection  model  should  be  capable  of  taking  decision- 
maker inputs  and  then  constructing  satisfactory  routes  for  that  decision  maker. 

Two  improvements  should  be  made  in  future  experiments.  First,  the 
dynamic  programming  algorithm  should  be  modified  to  find  the  least  threaten- 
ing route  for  the  decision-maker  inputs  obtained  in  these  experiments.  In 
addition,  the  Phase  I experiments  should  include  additional  questions  to  indi- 
cate how  decision  makers  perceive  threat  when  multiple  route  segments  are 
considered  with  diverse  target  azimuths  and  range.  These  improvements  may 
produce  a model  which  generates  routes  that  are  acceptable  to  almost  all 
decision  makers,  and  some  decision  makers  may  even  prefer  the  model  routes 
to  their  own  routes . 

The  Route  Planning  Decision  Process 
As  indicated  at  the  beginning  of  this  report,  very  little  was  actually 
known  about  the  route  planning  decision  process  when  the  research  program 
was  initiated.  In  Chapter  1,  it  was  noted  that  quite  a few  route  planning  models 
have  been  developed  and  have  been  used  in  simulations  of  combat  operations. 
However,  the  only  model  known  to  have  been  the  object  of  a validation  effort 
is  the  DYNTACS  X ground  unit  route  selection  model  (Shrive,  1976).  Thus, 
the  characteristics  of  the  decision  process  itself  have  remained  a mystery. 

Furthermore,  very  little  was  actually  known  about  the  characteristics 
of  the  military  decision-maker,  at  least  with  respect  to  his  behavior  in  a 
complex  decision  under  pressure  of  the  type  represented  by  the  route  planning 
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task.  Studies  of  firing  decisions  conducted  by  researchers  such  as  Olson 
(1970)  and  Parry  (1971)  had  shed  some  light  on  these  characteristics,  but  it 
was  unknown  whether  the  results  were  applicable  in  the  more  complex  route 
planning  task. 

From  the  above  observations  it  would  appear  that  practically  any  in- 
formation about  the  route  planning  decision  process  would  represent  an  im- 
provement over  the  situation  that  existed  at  the  start  of  the  research  program. 
Therefore,  it  would  have  been  almost  impossible  for  the  program  to  have 
totally  failed  in  this  respect.  However,  it  would  also  be  unlikely  that  the 
program  could  yield  total  knowledge  of  the  decision  process  and  the  character- 
istics of  the  decision-makers  involved. 

Information  about  the  decision  process  itself  has  been  gathered  from 
two  sources.  First,  as  discussed  in  Chapter  5,  a major  source  of  information 
was  the  set  of  comments  offered  by  the  subjects  during  the  route  evaluation 
session  of  Phase  II  and  the  answers  provided  by  the  subjects  to  questions  posed 
in  the  Master  Information  Form  and  Critique  administered  during  Phase  I. 

The  other  major  source  of  information  was  the  ratings  given  to  route  alterna- 
tives during  Phase  II  and  Trial  Two.  From  these  ratings  various  character- 
istics of  the  process  can  be  imputed.  Unfortunately,  the  research  program 
has  not  been  designed  as  a psychophysical  study  to  concentrate  directly  on  any 
particular  facet  of  the  decision  process,  save  for  the  threat  assessment  exer- 
cises conducted  during  Phase  I.  Consequently,  conclusions  about  the  decision 
process  are  mainly  of  a qualitative  nature.  Nevertheless,  it  is  felt  that  this 


i 


type  of  result  is  still  valuable. 
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To  repeat  the  summary  presented  in  Chapter  5,  subject  comments  and 
questionnaire  responses  indicate  that  pilots  select  their  routes  to  minimize 
exposure  to  enemy  weapons  and  maximize  the  element  of  surprise  during  an 
attack.  During  enroute  travel  they  fly  as  close  to  the  ground  as  possible  and 
will  go  to  any  length  to  avoid  being  intervisible  with  the  enemy.  If  intervisi- 
bility is  unavoidable  they  try  to  choose  a path  that  is  as  short  as  possible  and 
as  far  as  possible  from  the  enemy  weapon.  For  some,  even  an  exposure  time 
that  is  less  than  the  reaction  time  of  the  enemy  weapon  is  considered  dangerous. 

Proceeding  further,  pilots  obviously  consider  the  planning  of  the  firing 
activ’  be  the  most  important  aspect  of  preparing  a route  plan.  They  have 
and  fairly  consistent  criteria  for  selecting  a mode  of  fire  to  em- 
i 'j  in  a given  tactical  situation.  The  same  can  be  said  for  the  criteria  they 
use  in  selecting  a hover  fire  position.  For  example,  the  subjects  apparently 
favor  hover  fire  over  running  fire  unless  they  are  forced  into  a duel  by  some 
factor  such  as  terrain  (lack  of  covered  approaches)  or  weapon  effective  range 
(their  own  versus  the  enemy's).  They  apparently  would  prefer  to  deliver  fire 
from  a hover  fire  position  outside  the  range  of  the  enemy  weapon  if  possible. 
Furthermore,  in  choosing  a hover  fire  position  they  seek  a masked  avenue  of 
approach  from  which  they  may  "pop  up"  to  fire.  However,  they  try  to  avoid 
being  silhouetted  against  the  sky  from  this  position. 

From  the  above  summary  it  is  obvious  that  the  subjects  apparently 
agree  with  the  assumptions  that  were  made  in  developing  the  route  planning 
model  in  general.  Thus,  the  results  summarized  in  the  preceding  sections 

of  this  chapter  are  certainly  reasonable. 
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A line  of  sight  (LOS)  map  may  be  defined  as  a map  showing  areas  over  a 
battlefield  in  which  a helicopter  flying  a knap  of  the  earth  flight  profile  would  be 
intervisible  with  an  enemy  weapon  located  on  the  battlefield.  As  described  in 
Chapter  2,  such  maps  are  of  fundamental  importance  in  the  application  of  the 
route  planning  model  to  a route  planning  task.  Therefore,  in  this  appendix,  a 
description  of  the  methods  used  to  construct  LOS  maps  will  be  presented. 
Furthermore , the  methods  will  be  illustrated  with  material  actually  developed 
and  used  during  the  course  of  the  research. 

Problem  Definition 

Suppose  that  an  enemy  weapon  is  located  at  the  point  (XT , YT)  within  the 
area  of  a battlefield  defined  by  the  limits  (0  — XT  — XMAX)  and  (0  — YT  — YMAX). 
By  virtue  of  its  position  on  the  terrain,  the  weapon  has  an  elevation  ZT;  and, 
if  the  height  of  the  weapon  is  HT,  then  an  observer  located  at  (XT,  YT)  has  an 
overall  elevation 

HLOS  = HT+ZT.  (A.  1) 

Now,  suppose  helicopters  flying  knap  of  the  earth  are  assumed  to  fly  at 
one  of  two  elevations  above  the  terrain.  Over  open  terrain  this  elevation  is 
HOPEN,  while  over  wooded  regions  the  elevation  is  HOVER  + HTREE  where 
HTREE  is  the  height  of  trees  within  the  wooded  area.  If  at  a given  position 
(XO,  YO)  the  terrain  has  an  elevation  ZO,  then  the  overall  attitude  of  the  heli- 
copter flight  path  is 
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1 ZO  + HOVER  + HTREE  if  (XO,  YO)  is  within  a forest 
HFP  = ) 

( ZO  + HOPEN  otherwise.  (A. 2) 

The  point  (XT,  YT,  HLOS)  is  said  to  be  intervisible  with  the  point 
(XO,  YO,  HFP)  if  and  only  if  a straight  line  connecting  the  two  points  fails  to 
intersect  another  line  forming  the  profile  of  the  terrain  and  the  wooded  areas 
between  the  two  points  as  illustrated  in  Figure  A.l.  A LOS  is  said  to  exist  if 
the  two  points  are  intervisible. 

To  construct  a LOSmapof  the  type  used  by  the  route  planning  model, 
one  may  simply  specify  the  point  (XT,  YT,  HLOS)  as  one  point  in  the  inter- 
visibility relationship  and  then  analyze  all  other  points  (XO,  YO,  HFP)  for 
intervisibility,  where  0 — XO  — XMAX,  0 — YO  — YMAX  and  HFP  is  found  by 
the  relation  A. 2.  Of  course,  in  practice  one  would  analyze  only  a finite  num- 
ber of  points,  perhaps  a uniform  grid  of  points  covering  the  battlefield  with 
each  point  separated  from  neighboring  points  by  some  interval  A . 


Figure  A.l.  — Intervisibility  Relationships 
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Application 

For  the  research  program  it  was“cTecided  to  prepare  LOS  maps  for  an 
actual  piece  of  terrain  that  would  be  representative  of  battlefields  over  which 
engagements  of  the  type  being  studied  could  be  expected  to  occur.  Using  this 
approach  it  was  necessary  to  have  a method  for  determining  the  elevation  of 
the  terrain  at  any  battlefield  position.  It  was  also  necessary  that  a method 
exist  to  determine  whether  or  not  a given  point  on  the  terrain  is  contained 
within  the  boundaries  of  a forested  region.  Finally,  it  was  necessary  to  have 
a method  for  determining  whether  or  not  intervisibility  exists  between  any  two 
points  of  interest.  The  required  methods  are  discussed  in  the  following  para- 
graphs. 

Terrain. — The  battlefield  selected  for  use  in  the  research  program  was 
a five  by  ten  kilometer  area  from  the  Fulda  Gap  region  of  West  Germany. 
Arbitrarily,  XMAX  was  taken  as  the  ten  kilometer  dimension. 

Information  regarding  terrain  elevations  within  the  battlefield  area  was 
provided  by  data  prepared  by  the  U.  S.  Army  Map  Service.  In  raw  form,  these 
data  specify  the  elevation  of  terrain  at  ten  meter  intervals  in  both  X and  Y. 
However,  for  use  in  the  research  program,  the  data  were  screened,  and  a 
100  meter  interval  was  used.  Consequently,  5151  (101  x 51)  data  points  were 
required  to  specify  the  battlefield  terrain  contour.  Figure  A.  2 presents  a 
contour  map  of  the  battlefield  that  was  prepared  by  computer  using  the  elevation 
data  described  above'. 


2‘)0 
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Now,  the  battlefield  surface  was  represented  as  a continuous  surface 
by  the  method  illustrated  in  Figure  A. 3.  This  method,  discussed  in  detail  by 
Clark  (1969a),  allowsthe  terrain  elevation  of  any  point  (X,  Y)  within  the  battle- 
field area  to  be  determined.  The  ten  thousand  planar  surfaces  in  the  shape  of 
triangles  give  an  adequate  approximation  to  the  actual  terrain  surface . 


Figure  A. 3.  — Terrain  Representation 

Forests. — Situated  throughout  the  selected  battlefield  area  are  forested 
regions  composed  of  both  conifer  and  deciduous  vegetation.  By  photo  interpre- 
tation techniques,  Professor  Olin  W.  Mintzer  of  The  Ohio  State  University 
Department  of  Civil  Engineering  was  able  to  determine  that  the  average  height 
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of  trees  within  these  forests  is  approximately  twenty  meters  (HTREE  = 20). 1 
He  was  also  able  to  accurately  plot  the  boundaries  of  these  forests  within  the 
battlefield  area. 

Using  the  data  compiled  by  Professor  Mintzer,  a method  was  developed 
for  representing  forested  regions  within  the  simulated  battlefield  area.  First, 
however,  it  was  determined  that  the  actual  distribution  of  forested  regions  over 
the  battlefield  detracted  from  the  overall  worth  of  the  area,  at  least  as  far  as 
the  research  program  was  concerned.  For  example,  in  many  areas  too  much 
of  the  terrain  was  covered  by  forested  regions.  More  open  terrain  was  desired 
to  provide  areas  conducive  to  the  conduct  of  armored  operations. 

Therefore , it  was  decided  to  slightly  modify  the  actual  boundaries  of  the 
forested  regions  in  an  attempt  to  enhance  military  characteristics  of  the  terrain 
and  hence  to  increase  the  value  of  the  research.  In  certain  areas  forested  re- 
gions were  deleted  to  provide  more  open  terrain  for  the  reasons  cited  above. 

In  other  areas,  forest  boundaries  were  altered  to  better  define  avenues  of 

approach  to  and  fields  of  fire  from  positions  that  could  be  identified  as  key 
o 

terrain.  Finally,  the  boundaries  of  the  forested  regions  were  simplified  to 

^By  private  communication.  Professor  Mintzer  stated  that  conifer  vege- 
tation was  on  the  order  of  50-60  feet  in  height  while  the  older  deciduous  trees 
were  approximately  60 feet  tall.  The  forests  are  marked  by  uniform  vegetation 
height. 

2 

"Key  terrain  is  any  locality  or  area  the  control  of  which  affords  a 
marked  advantage  to  either  combatant.  Key  terrain  must  be  seized,  neutralized, 
or  otherwise  controlled  to  deny  its  use  by  the  enemy  or  permit  its  use  by  friend- 
ly forces."  Headquarters,  Department  of  the  Army.  Armor  Operations, 

FM  17-1,  October  1966,  p.  14. 
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facilitate  their  representation.  The  final  result  is  indicated  in  Figure  A. 4. 


The  method  used  to  represent  forests  is  as  depicted  in  Figure  A. 5. 
First,  the  battlefield  is  divided  into  fifty  strips  100  meters  wide  and  parallel 


to  the  X axis.  The  boundaries  of  the  strips  are  then  numbered  from  one  to 
fifty-one  starting  with  the  boundary  at  Y = 0.  Next,  the  presence  or  absence  of 
forests  along  any  strip  boundary  I forms  intervals  whose  end  points  J are  num- 
bered from  left  to  right  with  as  many  as  four  intervals  being  allowed 
(J  = 1,  2,  . .. , 8)  . Such  a limit  was  determined  to  be  satisfactory  by  analyz- 
ing the  modified  forest  boundaries  shown  in  Figure  A. 4. 

Finally,  the  X coordinates  of  the  endpoints  of  intervals  containing 
forests  are  recorded  in  the  matrix  IT  (I,  J)  whose  entries  are  defined  as  follows: 


IT  (I,  J) 


[£] 


where  [A]  indicates  the  greatest  integer  equal  to  or  less  than  A.  Thus,  the  X 
coordinate  of  a given  interval  end  point  (I,  J)  can  be  recovered  from  IT  (I,  J) 
with  an  error  of  less  than  100  meters. 

As  an  example  of  the  procedure,  suppose  the  4^  strip  boundary  is  being 
analyzed  (Y  - 300).  From  Figure  A.  5 it  is  seen  that  there  are  three  forested 
intervals.  The  first  occurs  between  X = 1520  and  X = 3140,  the  second  between 
X = 3330  and  X = 4510,  and  the  third  between  X = 8910  and  X = 10000.  Thus, 
the  entries  in  IT  for  1 = 4 arc: 


IT  (4,  1)  = 16 
IT (4,  2)  = 32 
IT  (4,  3)  = 34 


IT(4,  4)  = 46 
IT(4,  5)  = 90 
JT(4,  6)  =101 
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Note  that  if,  for  a given  I,  there  are  no  forested  intervals,  then  IT  (I,  1)  - 101 
can  be  used  to  denote  this  fact. 

Figure  A.  6 presents  a plot  of  the  forested  regions  as  described  by  the 
method  outlined  above.  The  plot  was  prepared  by  computer  and  shows  that  the 
method  results  in  a fairly  accurate  representation.  The  forst  boundaries 
appear  smooth  and  the  shapes  closely  approximate  those  of  the  actual  forested 
areas  (see  Figure  A. 4). 

Now,  to  determine  whether  a given  point  (X,  Y)  lies  within  a forested 
region,  the  following  procedure  was  used.  First,  the  (X,  Y)  position  is  mea- 
sured to  the  nearest  100  meters ; that  is 


where  £ A j indicates  the  largest  integer  equal  to  or  less  than  A.  Next,  the 
modified  coordinates  are  transformed  into  integers  to  be  used  with  the  matrix 
IT  (I,  J);  that  is 


Y'  U i 
+ 1 

where  ^Aj  is  defined  as  above.  Finally,  the  following  logic  is  used  to  deter- 
mine whether  or  not  (X,  Y)  lies  within  a forested  region: 


IT*  = 


X’ 

100 


I 

I 

I 
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-Computer  Prepared  Plot  of  Forested  Regions 


I 

I 

I 

I 


if  IT  (I',  1)— IT'  ^IT(r,  2), 

or  IT(T,  3)  — IT'  ^IT(r,  4), 
or  IT  (I' , 5)  — IT'  — IT  (I' , 6), 
or  IT(I\  7)  — IT'  — IT(T,  8), 

then  (Y,  Y)  is  in  a forest;  otherwise  (X,  Y)  is  not  in  a forest. 

Using  the  above  procedure  the  point  (X  = 8860,  Y = 340)  would  be  con- 
sidered in  a forest  while  the  point  (X  = 8840,  Y = 340)  would  not  be.  However, 
examination  of  Figure  A.  4 reveals  that  both  points  are  in  actuality  outside  the 
forest  boundaries.  Nevertheless  errors  of  this  magnitude  were  considered 
acceptable. 

LOS  Maps  for  Phase  II  Experiment 

The  Phase  II  Experiment  described  in  Chapter  4 and  Appendix  D required 
that  four  unique  tactical  situations  be  developed,  with  each  situation  involving 
two  enemy  weapons  located  at  separate  positions  on  the  battlefield.  Analysis  of 
the  terrain  revealed  that  the  four  situations  could  be  represented  satisfactorily 
by  using  only  a total  of  five  separate  weapon  positions.  The  five  positions 
selected  are  outlined  in  Table  A.  1. 


Table  A.  1 
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The  four  tactical  situations  were  developed  by  forming  pairs  from  the 
five  weapon  positions  as  indicateu  in  Table  A.  2.  By  comparing  the  terrain  map 
in  Figure  A.  2 with  the  situations  indicated  in  Table  A.  2,  it  may  be  verified  that 
each  situation  is  representative  of  one  that  might  be  found  in  combat.  All  posi- 
tions are  seen  to  occupy  key  terrain.  Furthermore,  in  each  situation  the  two 
weapon  positions  are  mutually  supportive.  That  is,  each  weapon  position  could, 
if  required,  provide  covering  fire  for  the  other  position. 

Now,  LOS  maps  were  required  to  depict  intervisibility  relationships  for 
each  of  the  four  tactical  situations.  Of  course,  this  meant  that  the  LOS  maps  to 
be  developed  would  actually  be  composite  maps  formed  by  overlaying  the  two 
maps  associated  with  the  two  weapon  positions  considered  in  each  tactical 
situation.  Thus,  five  individual  LOS  maps  corresponding  to  the  five  weapon 
positions  of  Table  A.  1 were  required.  The  maps  appear  in  Figures  A.  7 through 
A.  11,  with  shaded  areas  indicating  intervisibility. 

Figures  A.  7 through  A.  11  were  prepared  with  the  aid  of  a computer  pro- 
gram developed  specifically  for  the  task.  As  a start  point  for  the  program,  one 


Table  A.  2 

Tactical  Situation  Summary 


Tactical 

Situation 

Weapon  Position  Number 

Weapon  2 

A 

1 

2 

B 

4 

3 

C 

3 

5 

D 

4 

5 
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Figure  A.  7.  — LOS  Map  for  Weapon  Position  1 


Figure  A.  8.  — LOS  Map  for  Weapon  Position  2 


Figure  A.  10.  — LOS  Map  for  Weapon  Position  4 


Figure  A.  11.  — LOS  Map  for  Weapon  Position  5 


of  the  weapon  positions  specified  in  Table  A.  1 is  taken  as  the  point  (XT,  YT)  for 
which  the  LOS  map  is  desired.  Next,  the  elevation  of  the  terrain  ZT  at  (XT,  YT) 
is  found  by  interpolating  (see  Figure  A.  3)  upon  the  terrain  data  described 
aarlier,  and  the  height  of  the  observer  HLOS  is  determined  using  equation  A.  1. 
Then,  a uniform  grid  of  points  is  laid  down  over  the  battlefield  with  each  point 
separated  from  its  neighbors  by  100  meters.  Each  grid  point  (XO,  YO)  is  then 
analyzed  to  determine  if  a LOS  exists  to  the  point  (XT,  YT,  HLOS).  This  pro- 
cedure consists  first  of  an  interpolation  to  find  the  terrain  elevation  ZO.  Then 
the  procedure  outlined  earlier  is  applied  to  determine  whether  (XO,  YO)  lies 
within  a forest.  Next,  the  appropriate  altitude  of  the  knap  of  the  earth  flight 
profile  HFP  is  determined  using  equation  A.  2.  Finally,  an  algorithm  developed 
by  Clark  (1969a)  is  used  to  determine  whether  or  not  the  point  (XT,  YT,  HLOS) 
is  intervisible  with  the  point  (XO,  YO,  HLOS).  The  algorithm  uses  the  planar 
representation  (Figure  A.  3)  of  the  terrain  elevation  data  to  determine  whether 
terrain  (or  woods)  intersects  a line  between  the  two  points. 

Once  each  grid  point  has  been  analyzed  for  intervisibility,  data  for  con- 
struction of  the  LOS  map  a re  available.  Special  library  programs  are  used  to 
generate  an  instruction  deck,  which  in  turn  may  be  used  to  drive  a plotter.  The 
plotter  then  prepares  a drawing  of  the  type  illustrated  by  Figures  A.  7 through 


Exhibit  B.  1 

Experimental  Session  Plan 

Part  A Preliminary  Session  to  Enlist  Volunteers 

A.l  Present  discussion  of  general  intent  of  the  experiment 
A. 2 Pass  out  Volunteer  Information  Forms  (VIF) 

A.  3 Collect  completed  VIF  from  each  prospective  subject 

Part  B Phase  I Data  Collection  Session 

B.  1 Present  introduction  and  scenario  description 

■ 

B.  2 Pass  out  Enemy  Weapon  Brochure  (EWB) 

B.3  Present  discussion  of  EWB 

B.4  Pass  out  Master  Information  Form  (MIF) 

B.5  Present  instructions  for  answering  MIF  questions  I,  II  and 
III 

B.6  Pause  to  permit  subjects  to  answer  MIF  questions  I,  II  and 

in 

B.7  Present  introduction  to  exposed  threat  evaluation  subsession 

B.8  Pass  out  Maximum  Threat  Assessment  Form  (MTAF) 

B.9  Present  instructions  for  using  MTAF  to  locate  maximum 
threat  situation(s) 

B.10  Present  instructions  for  using  prepunched  computer  data 
cards  to  record  exposed  threat  values 

B.ll.a  Present  exposed  threat  assessment  slides 

B.ll.b  Pause  on  each  slide  to  permit  subjects  to  record  answer 
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I 

I 

B.12.a  Present  instructions  for  using  MTAF  to  record  maximum 
threatening  ranges 

B.12.b  Present  example  slide 

B.13  Pause  to  permit  subjects  to  record  maximum  threatening 
ranges 

B.  14  Present  instructions  for  determining  unexposed  threat 
equivalent  data 

B.15.a  Present  unexposed  threat  equivalent  slides 

B.15.b  Pause  on  each  slide  to  permit  subjects  to  record  answer 

B.16.a  Present  instructions  for  answering  MIF  question  IV 

B.IG.b  Present  example  slide 

B.17  Pause  to  permit  subjects  to  record  answer 

B.18  Collect  all  material  from  each  subject 
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Summary  Text  of  Session  Plan  Item  A.l 
Intent  of  Experiment 

The  Ohio  State  University  Systems  Research  Group  (SRG)  has  been  con- 
tracting almost  continuously  with  the  U.  S.  Army  Combat  Developments  Com- 
mand since  1950.  This  association  commenced  with  studies  of  armored  war- 
fare conducted  for  the  Armor  Agency  at  Fort  Knox,  and  more  recently,  has 
been  carried  on  by  contracts  with  the  Systems  Analysis  Group  at  Fort  Belvior. 
Contract  work  of  a similar  nature  has  also  been  performed  for  the  U.  S.  Army 
Missile  Command  since  19G6. 

In  1965,  SRG  commenced  work  to  build  a simulation  model  of  armored 
combat  that  has  become  known  as  DYNTACS.  This  model  is  designed  to  permit 
studies  of  the  operational  performance  of  armored  units  in  attack,  defense  and 
delaying  engagements.  Therefore,  DYNTACS  takes  into  consideration  many  of 
the  interactions  that  exist  between  the  physical  performance  of  weapons,  the 
terrain  and  environment  in  which  they  operate,  and  the  tactics  with  which  they 
are  employed. 

DYNTACS  is  run  on  a large-scale  digital  computer  in  a completely 
"hands-off"  mode.  Thus,  the  various  combat  decisions  made  by  individual  tank 
commanders  and  by  unit  commanders  must  be  modeled  explicitly  and  internal 
to  DYNTACS.  Examples  of  such  decisions  are  as  follows: 
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At  whom  should  a tank  crew  fire  ? 

When  should  fire  take  place? 

What  ammunition  should  be  used? 

What  route  to  the  objective  should  the  attacking  unit  choose  ? 

These  decisions  models  are  supported  with  emperical  data  whenever  possible. 

In  1970,  a very  detailed  model  of  helicopter  operations  was  incorporated 
into  DYNTACS  for  the  Systems  Analysis  Group  at  Fort  Belvoir.  This  model  is 
designed  to  represent  combat  activities  of  helicopter  units  providing  direct 
aerial  fire  support  for  tank  units  participating  in  the  ground  engagement. 
Therefore,  the  helicopter  units  operate  in  the  conventional  warfare  environ- 
ment implied  by  the  types  of  engagements  represented  in  DYNTACS.  The  model 
must  predict  operational  performance  of  helicopter  units  with  the  same  degree 
of  accuracy  as  that  which  is  produced  by  the  model  of  ground  unit  performance. 
Thus,  the  interactions  that  exist  between  helicopter  weapon  performance,  ter- 
rain over  which  the  helicopter  units  operate,  and  the  tactics  with  which  they  are 
employed,  must  be  predicted  accurately. 

However,  at  the  time  the  helicopter  operations  model  was  designed, 
detailed  descriptions  of  helicopter  tactics  to  be  used  in  a conventional  warfare 
environment  were  not  available.  Gross  approximations  were  derived  from 
stated  doctrine  and  incorporated  into  the  tactical  decision  models.  However, 
no  explicit  emperical  evidence  existed  to  support  more  detailed  representations 
of  helicopter  tactics.  For  example,  the  route  to  be  used  by  a unit  attacking  a 
hard,  highly  lethal  weapon  is  determined  by  a fairly  unsophisticated  model  that 
is  based  upon  doctrine  but  is  largely  unsupported  by  actual  data. 
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This  brings  us  to  the  problem  at  hand.  SRG  now  intends  to  refine  some 
of  the  existing  helicopter  tactical  decision  models  by  gathering  experimental 
evidence  from  qualified  Army  aviators  and  by  making  model  modifications  as 
required.  Therefore,  SRG  is  seeking  as  many  aviator  volunteers  as  possible 
to  participate  in  an  experiment  to  be  held  here  at  Fort  Rucker  tomorrow 
(December  28,  1971).  This  experiment  will  be  conducted  in  a classroom 
or  other  suitable  indoor  location  to  be  announced,  it  will  be  strictly  a pencil 
and  paper  exercise,  and  it  will  last  approximately  two  hours.  During  the 
experiment  volunteer  subjects  will  be  asked  to  make  simple  value  judgments 
and  to  select  routes  in  various  hypothetical  situations.  The  piloting  skill  of 
volunteer  aviator  subjects  will  never  be  in  question;  only  their  opinions  as 
qualified  military  decision  makers  will  be  solicited.  In  fact,  during  the  exper- 
iment and  afterward,  subjects  will  never  be  identified  individually.  All  data 
collected  will  be  handled  with  strictest  confidence. 

Please  fill  out  the  Volunteer  Information  Forms  I am  passing  around 
now.  If  you  wish  to  volunteer  simply  indicate  your  desire  on  the  form  and 
remain  seated  for  further  instructions.  Others  electing  not  to  volunteer  should 
fill  out  the  form  and  hand  it  to  me  before  leaving.  I appreciate  your  cooperation 
and  thank  you  on  behalf  of  the  Systems  Analysis  Group  whose  support  makes 


812 


this  study  possible. 
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Example  of  Session  Plan  Item  A.  2 
Volunteer  Information  Form 


PLEASE  COMPLETE  ALL  ANSWERS  ON  THIS  FORM 
L Personal  Data: 

Ao  Name — ...  — B-  Rank  _ 

Last  First  Middle 

C.  Date  of  birth 

Month  Year 

D.  Active  service . 

Years/Months 

E.  Number  of  months  at  present  duty  station  . 


IL  Total  Rotary  Wing  Experience: 

A.  Approximate  total  flight  time 

Hours 


B.  Approximate  flight  time  in  a combat  zone . 

Hours 

1.  Location  of  combat  zone 

2.  Approximate  dates  in  combat  zone 

Month/Year 


J3L  Rotary  Wing  Combat  Experience: 


A. 


Approximate  flight  time  (hours) 

1.  Gunship 

2„  Troop  carrier 

3.  Observation _ 

4..  Med  evac 

5,  Other  (specify) 
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B.  Vehicle  types  (Huey,  Cobra,  ®te.) 

1.  Gunrhip  — 

2.  Troop  carrier — 

3„  Observation — 

4.  Med  ovac  _ — 

5,  Other  (specify) 

C.  What  is  the  most  potent  air  defense  weapon  you  have  ever 

encountered  ? 

Were  you  engaged  ? . . 

Did  you  engage?  

D„  Whafc  types  of  weapons  have  you  fired  in  combat? 


Rockets 

7,  62  mm 

Offcer  (specify) 


GrenadeB 

Missiles 


Have  you  ever  engaged  a tank  or  other  hard,  point  target  in 
combat? 

If  yes  what  types  of  weapons  were  used? 


Miscellaneous  (IMPORTANT): 

A.  Have  you  ever  been  instructed,  formally  or  otherwise,  in  Hie 
tactics  and  doctrine  of  helicopter  employment  in  conventional, 
mid-intensity  warfare  ? 

If  yes,  plea»8  describe  the  circumstances. 


Availability: 

Do  you  volunteer  to  participate  in  the  described  experiment  (of 
approximately  two  hours  duration) 


I 

I 
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Summary  Text  of  Session  Plan  Item  B.  1 
Introduction  and  Scenario  Description 

Good  morning  gentlemen.  As  indicated  yesterday  we  are  going  to  be 
together  for  about  two  hours  today,  and  during  this  time  we  are  going  to  be 
concentrating  on  the  mental  process  by  which  you  select  routes  in  combat. 

You  have  been  selected  to  participate  because  each  one  of  you  has  faced 
the  enemy  in  combat  and  has  experienced  the  problem  of  selecting  a route  to  be 
followed  while  attacking  a target.  Let  me  emphasize  again  that  there  are  no 
right  or  wrong  answers  to  any  of  the  questions  that  I will  ask  you  this  morning. 
Your  skill  as  an  aviator  is  not  in  question.  Instead,  I wish  only  to  obtain  your 
opinions  as  highly  qualified  military  decision  makers . The  data  that  I collect 
will  be  held  in  strictest  confidence. 

Now,  to  give  us  a frame  of  reference,  let  me  describe  a tactical  scenario 
that  gives  rise  to  the  type  of  route  selection  task  in  which  I am  interested  in 
this  experiment.  This  scenario  is  typical  of  one  which  you  might  actually 
experience  in  a mid-intensity  conventional  warfare  environment.  Also,  it  is 
the  scenario  that  you  should  have  in  mind  each  time  you  feel  a need  to  relate 
to  the  "real  world"  in  order  to  answer  any  of  the  questions  which  I will  ask 
you  later  on  during  the  session. 

Consider  a fire  team  of  Army  attack  helicopters  whose  mission  it  is  to 
provide  immediate  direct  aerial  fire  support  to  an  armored  battalion  in  a mid- 


I 
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intensity  conventional  warfare  situation.  The  battalion  has  been  fighting  a ue- 
laying  engagement  and  has  fallen  back  through  a succession  of  defensive  posi- 
tions over  a period  of  several  days.  The  fire  team  has  assisted  in  the  delay  by 
providing  harassing  fires  in  the  enemy  flanks  and  by  serving  as  part  of  the  base 
of  fire  used  to  cover  each  withdrawal.  The  enemy  has  occupied  and  consolidated 
his  hold  on  each  position  prior  to  initiating  the  attack  on  additional  objectives. 

Now,  the  friendly  forces  have  begun  to  contain  the  attack  and  the  first 
major  thrust  to  recapture  lost  territory  is  to  be  mounted.  As  the  battalion  com- 
mences its  assault  the  enemy  is  able  to  bring  elements  of  the  maneuver  force 
under  very  heavy  fire  and  before  long  an  entire  company  is  pinned  down.  The 
battalion  commander  is  faced  with  the  alternatives  of  diverting  part  of  his  force 
to  attack  the  enemy  position  from  its  flank  or  of  requesting  some  form  of  fire 
support  to  neutralize  or  destroy  the  enemy  position.  He  decides  to  utilize  the 
attack  helicopter  team. 

The  helicopter  team  has  been  waiting  in  anticipation  of  just  such  a mis- 
sion. It  has  been  sitting  on  the  ground  to  the  immediate  rear  of  the  supported 
battalion  and  is  fully  armed  and  fueled.  The  team  consists  of  two  advanced 

fire  support  helicopters  with  point  fire  missiles  as  their  primary  anti-armor 
armament. 


By  radio,  the  battalion  commander  gives  the  team  leader  a description 
of  the  target  and  a briefing  regarding  the  disposition  of  friendly  forces  and  the 
suspected  locations  of  other  hostile  forces.  The  mission  for  the  helicopter 
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As  described,  the  enemy  position  is  occupied  by  two  tanks,  an  armored 
personnel  carrier  and  a self  propelled  anti-aircraft  gun.  The  A PC  serves  as 
the  transport  vehicle  for  two  dismounted  crews,  each  equipped  with  a medium 
anti-tank  weapon.  The  anti-aircraft  gun  provides  defense  for  the  position  from 
aerial  attack. 

In  addition  to  the  weapons  at  the  target  position  there  is  a reported  enemy 
anti-aircraft  gun  located  to  the  rear  of  the  target  position  but  close  enough  to 
provide  supporting  fire  against  any  helicopters  that  might  attack.  This  weapon 
is  radar  directed  and  provides  a large  volume  of  highly  lethal  fire. 

Upon  receipt  of  the  mission  request  the  fire  team  leader  marks  the  posi- 
tions of  the  target  and  the  supporting  anti-aircraft  gun  on  his  map.  He  then 
commences  preparation  of  a plan  of  attack  for  his  fire  team.  He  must  select  a 
route  to  the  vicinity  of  the  target  and  a return  route  to  be  used  upon  completion 
of  the  mission.  He  must  also  select  the  method  by  which  the  target  will  be  at- 
tacked. The  entire  attack  plan  is  recorded  on  the  map  and  is  conveyed  to  the 
other  team  member  prior  to  take-off. 

Now,  the  attack  plan  serves  only  as  a guide  during  the  mission.  Ob- 
viously, as  the  situation  unfolds,  as  the  enemy  takes  action,  as  the  terrain 
actually  becomes  visible,  as  ordnance  is  delivered,  the  plan  will  change.  The 
team  leader  must  make  decisions  during  the  entire  mission  that  affect  the  out- 
come of  the  mission.  However,  a very  important  part  of  the  mission  is  the 
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initial  plan  and  it  is  this  planning  operation  that  I have  referred  to  earlier  as 
the  route  selection  process. 

During  the  experiment,  details  of  the  missions  you  are  asked  to  visu- 
alize will  change . That  is , the  type  of  enemy  weapon  to  be  attacked  and  the 
type  of  emeny  weapon  in  support  of  the  target  will  not  be  the  same  as  described 
above.  However,  the  overall  tactical  situation  and  the  route  planning  task  you 
are  given  will  always  remain  as  outlined. 


Exhibit  B.  5 


Example  of  Session  Plan  Item  B.  2 

Enemy  Air  Defense  Weapons 
(Unclassified — for  illustrative  purposes  only) 

L General  Description 

The  REDEYE  is  an  all  arms,  man-portable,  shoulder-launched,  heat 
seeking  guided  missile  system.  Each  launch  team  of  two  men  is  supplied  with 
a basic  load  of  six  rounds. 

The  23  mm  quad  self-propelled  antiaircraft  weapon  is  a light  antiair- 
craft weapon  system  consisting  of  a radar- directed,  quad- mount,  23  mm  auto- 
matic weapon  mounted  on  a fully- tracked  chassis.  The  weapon  is  manned  by 
a crew  of  four  and  is  supplied  with  a basic  load  of  4000  rounds. 

The  14.  5 mm  quad  antiaircraft  weapon  is  a light  antiaircraft  weapon 
system  consisting  of  a quad- mount,  14.  5 mm  automatic  weapon  with  ring  sight 
mounted  on  a fully-tracked  armored  personnel  carrier.  The  weapon  is  manned 
by  a crew  of  three  and  is  supplied  with  a basic  load  of  8000  rounds. 

II.  Performance  Characteristics 
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Projectile 


3.0 


Type 
HE  I 


Fuzing 

Impact 


B 1.0  API  Impact 

HE  I 

C 0.3  API  Impact 


API  - Armor  piercing  incendiary 
HE  I - High  explosive  incendiary 
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Exhibit  B.  6 

Summary  Text  of  Session  Plan  Item  B.  3 

Discussion  of  Enemy  Weapon  Brochure 

The  enemy  weapon  characteristics  outline  I have  passed  out  presents 
detailed  data  on  the  enemy  weapons  we  will  be  concerned  with  during  the  re- 
mainder of  the  session  this  morning.  You  will  note  that  the  detailed  character- 
istics have  been  revised  from  official  estimates  so  that  we  can  avoid  classified 
discussions.  If  you  are  already  familiar  with  some  of  these  weapons,  I ask 
that  you  forget  what  you  know  and  consider  the  information  on  these  sheets  as 
being  official.  It  is  particularly  important  that  you  follow  this  suggestion  so 
that  I may  consider  each  one  of  you  equally  knowledgeable  with  respect  to  the 
enemy  weapons  under  consideration.  Please  feel  free  to  ask  any  questions  you 
might  have  concerning  the  data  on  these  sheets. 


Exhibit  B.  7 


Example  of  Session  Plan  Item  B.  4 
MASTER  INFORMATION  FORM 

L Subject  Number Subject  Name 

Last  First 

IL  Vehicle  Type  

IE.  Speed  Information 

A.  During  knap  of  the  earth  flight  to  a firing  position 

1.  Average  speed  (knots) 

2.  Range  of  variation  ( * knots  ) 

3.  Additional  comments 

B.  During  firing  run 

1.  Average  speed  (knots)  

2.  Range  of  variation  ( ^ knots  ) 

3.  Describe  the  manner  in  which  speed  changes  during  various 
stages  of  a firing  run. 

4.  Additional  comments 


♦Record  the  vehicle  type  in  which  you  would  feel  most  comfortable  were  you  to 
be  faced  with  a mission  of  the  type  described  in  the  introduction. 
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IV.  Route  Rating 

Rank  Route 

1  

2  

3 

4 


VI.  Preferred  Mailing  Address  and  Phone  Number 


I 
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Exhibit  B.  8 


Summary  Text  of  Session  Plan  Item  B.  7 
Introduction  to  Exposed  Threat  Evaluation 

Gentlemen,  during  the  remainder  of  the  session  we  are  going  to  be  dis- 
cussing your  feelings  about  the  enemy  weapons  that  were  described  to  you 
earlier.  In  particular,  we  are  going  to  discuss  the  "threat"  you  would  feel 
were  you  to  be  exposed  to  the  weapons  under  various  tactical  conditions.  This 
is  the  reason  we  spent  a considerable  amount  of  time  in  becoming  familiar  with 
the  weapons. 

Now,  what  do  we  mean  by  "threat.  " Suppose  you  were  flying  at  tree  top 
level  and  you  suddenly  realized  that  for  the  next  fifteen  (15)  seconds  you  were 
going  to  be  exposed  to  an  enemy  23  mm  weapon.  Furthermore,  suppose  you 
knew  that  the  bearing  from  your  helicopter  to  this  weapon  would  be  ninety  (90) 
degrees  and  the  range  would  be  one-thousand  (1000)  meters.  The  "threat"  we 
are  interested  in  is  that  feeling  in  the  pit  of  your  stomach  you  would  have  when 
you  first  realized  the  experience  you  were  about  to  endure.  I believe  this  feel- 
ing is  commonly  referred  to  as  "pucker  factor"  among  military  men  for  reasons 
we  need  not  pursue. 

Presumably,  we  could  change  any  of  the  situation  variables  in  the  ex- 
ample outlined  above  and  your  feeling  of  threat  would  also  change.  For  example, 
the  exposure  time  could  go  up  or  down,  the  type  of  enemy  weapon  could  change, 
or  the  relative  bearing  and  range  to  the  weapon  could  be  altered.  For  each  com- 
bination of  these  variables  you  might  register  a different  feeling  of  threat. 
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In  the  remainder  of  this  session  we  are  going  to  examine  a variety  of 
exposure  situations  involving  the  enemy  weapons  discussed  earlier.  Using  a 
method  to  be  explained,  you  will  actually  attach  numbers  to  the  threat  you  feel 
in  each  situation  and  in  this  way  I will  obtain  part  of  the  data  required  in  my 
study  of  the  route  selection  process. 

Let  me  stress  again  that  in  registering  your  feelings  about  threat  you 
should  try  to  use  as  a frame  of  reference  the  weapon  characteristics  that  have 
been  outlined  for  you  today.  Please  try  to  refrain  from  interjecting  your  own 
private  knowledge  of  the  enemy  weapons.  We  wish  to  have  everyone  working 
from  the  same  information  base  in  this  experiment. 


32fi 


f 


Subject  Number MAXIMUM  DIFFICULTY  ASSESSMENT 


4 
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Exhibit  B.  9 

Example  of  Session  Plan  Item  B.  8 
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Exposure  Time:  5 seconds  Exposure  Time:  15  seconds  Exposure  Time:  45  seconds 


Subject  Number MAXIMUM  DIFFICULTY  ASSESSMENT 

WEAPON  TYPE:  23  mm 


Exhibit  B.  10 


Summary  Text  of  Session  Plan  Item  B.  9 
Instructions  for  Locating  Maximum  Threat  Situation 

The  Maximum  Threat  Assessment  Form  I have  just  handed  each  of  you 
presents  a summary  of  all  exposure  situations  in  which  we  are  interested.  You 
will  note  there  is  one  page  for  each  weapon,  and  as  you  go  from  one  panel  to 
another  on  a given  page  the  exposure  time  changes.  The  black  line  on  each 
panel  represents  your  flight  path,  while  the  red  arrow  indicates  an  exposed 
segment  which  you  are  anticipating.  The  length  of  the  arrow  is  a visual  clue  to 
the  amount  of  exposure. 

The  small  circles  represent  all  the  possible  positions  that  the  enemy 
weapon  may  occupy  relative  to  your  flight  path.  On  each  panel  there  are  eleven 
(11)  such  positions.  Please  take  a few  moments  to  examine  this  form  and  feel 
free  to  ask  any  questions  you  might  have  regarding  the  information  presented. 
(Pause) 

Please  examine  the  first  panel  on  the  first  page.  Of  the  eleven  enemy 
positions,  which  position  do  you  feel  is  the  most  threatening  as  far  as  you  are 
concerned?  That  is,  in  which  position  would  you  least  like  to  find  the  14.  5 mm 
weapon  situated?  Please  indicate  the  position  with  a check  mark.  If  there  is 
more  than  one  position,  simply  mark  each  one. 

(Pause) 

Now,  examine  the  second  and  third  panels  and  repeat  the  process  you  used 
on  the  first  panel.  Examine  each  panel  independently  of  the  others. 
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(Pause) 

When  you  have  finished  the  third  panel,  I want  you  to  consider  the 
checked  positions  on  all  three  panels  simultaneously.  Of  the  positions  checked 
which  one(s)  is  the  most  threatening?  Please  indicate  your  selection(s)  with  a 
circle  around  the  check  mark(s). 

(Pause) 

Now  repeat  the  above  process  for  the  second  page,  and  when  you  finish, 
proceed  to  the  third  page.  Consider  each  page  independently  of  the  other  two. 
(Pause) 

Finally,  I want  you  to  examine  the  circled  check  marks  and  determine 
that  position(s)  which  is  the  most  threatening.  That  is,  in  this  last  step  we  are 
locating  the  position(s)  that  is  the  most  threatening  of  all  ninety-nine  (99)  posi- 
tions. Place  a square  around  the  position  selected. 

(Pause) 

Now,  what  we  have  determined  is  the  most  threatening  enemy  position  of 
all  positions  that  we  will  examine  today.  Let  us  now  agree  that  when  we  encoun- 
ter this  situation  in  the  upcoming  presentation  of  exposure  situations,  we  will 
assign  a value  of  one-thousand  (1000)  to  the  situation  that  we  feel  most  threatening. 
Let  us  further  agree  that  if  we  feel  no  threat  whatsoever  in  a given  situation  we 
will  assign  a value  of  zero  to  the  threat  (0).  Thus,  all  situations  will  be  assign- 
ed a threat  value  that  lies  between  zero  (0)  and  one  thousand  (1000).  Please 
write  those  numbers  down  on  the  form  as  a reference,  and  keep  this  form  handy 


I 
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during  the  remainder  of  the  session. 
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Exhibit  B.  11 


Summary  Text  of  Session  Plan  Item  B.  10 
Instructions  for  Using  Data  Cards 

Now  that  we  have  defined  threat  and  have  developed  a method  to  record 
threat  as  a numerical  quantity,  we  are  going  to  put  the  method  to  use  by  showing 
a series  of  slides  depicting  various  exposure  situations.  As  each  slide  is  pre- 
sented you  will  record  the  threat  you  feel  according  to  the  method  that  we  have 
developed. 

Your  answers  are  to  be  placed  on  the  computer  data  cards  that  each  of 
you  have  in  your  supply  box.  You  will  notice  that  the  card  deck  consists  of  one 
hundred  eighteen  (118)  cards  and  that  all  the  cards  have  your  SUBJECT  NUMBER 
recorded  on  them.  The  cards  are  ordered  by  the  SLIDE  NUMBER  appearing  at 
the  center  of  each  card.  These  numbers  correspond  to  the  numbers  of  the 
slides  that  I am  going  to  present  to  you. 

As  each  slide  is  presented,  you  will  check  the  slide  number  on  the  screen 
against  the  slide  number  on  the  card.  Then,  after  studying  the  situation  depicted 
by  the  slide,  you  will  record  the  threat  you  feel  on  the  card.  Your  answer 
should  appear  in  the  clear  area  of  the  card  under  the  word  VALUE. 

As  each  slide  is  shown  I will  allow  ample  time  for  you  to  analyze  the 
situation  and  record  your  answer.  As  soon  as  you  complete  each  card  simply 
place  it  back  in  the  supply  box  so  that  it  cannot  be  referred  to  again. 
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Exhibit  B.  12 

Example  of  Cards  Used  in  Session  Plan  Item  B.  11 


As  seen  by  subjects 


Exhibit  B.  13 


Examples  of  Session  Plan  Item  B.  11 


WEAPON  TYPE 23 

EXPOSURE  TIME 15 

MINIMUM  RANGE 2400 

MAXIMUM  RANGE 2420 

1 


mm 

seconds 

m 

m 
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WEAPON  TYPE 14.5  mm 

EXPOSURE  TIME 45  seconds 

MINIMUM  RANGE 0 m 

MAXIMUM  RANGE 1530  m 


I 

1 
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WEAPON  TYPE 23 

EXPOSURE  TIME 5 

MINIMUM  RANGE 2330 

MAXIMUM  RANGE 2470 


ram 

seconds 

ra 

m 
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WEAPON  TYPE REDEYE 

EXPOSURE  TIME 15  seconds 

MINIMUM  RANGE 0 in 

MAXIMUM  RANGE 310  m 

4- 
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WEAPON  TYPE 14.5 

EXPOSURE  TIME 5 

MINIMUM  RANGE 1100 

MAXIMUM  RANGE 1300 

5 
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Exhibit  B.  14 

Definition  of  Situations  Presented  in 
Session  Plan  Item  B.  11 


Weapons  (W) 

Level  (I) 

Type 

1 

2 

3 

14.5  mm 
23  mm 
REDEYE 

Exposure  Time  (T) 


Range  (R) 


Level  Value 

(K)  (meters) 


Level  (J) 

Value  (seconds) 

1 

5 

2 

15 

3 

45 

Relative  Bearing  (<(>) 

Level  (L) 

Value  (degrees) 

1 

0 

2 

45 

3 

90 

4 

135 

5 

180 

Situation  Number 

N = 33  (1-1)  + 11  (J-l)  + 5 (K-l)  + L 
for  K = 1,  2 
N = 33  (1-1)  + 11  J 

for  K = 3 
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Exhibit  B.  15 


Ordering  of  Situations  Presented  in  Session  Plan  Item  B.  11 


Slide 

Situation 

Levels 

W 

n 

R 

<t> 

1 

47 

2 

2 

1 

3 

2 

28 

1 

3 

2 

1 

3 

37 

2 

1 

1 

4 

4 

88 

3 

2 

3 

- 

5 

5 

1 

1 

1 

5 

6 

19 

1 

2 

2 

3 

7 

67 

3 

1 

1 

1 

8 

90 

3 

3 

1 

2 

9 

10 

1 

1 

2 

5 

10 

64 

2 

3 

2 

4 

11 

1 

1 

1 

1 

1 

12 

15 

1 

2 

1 

4 

13 

53 

2 

2 

2 

4 

14 

91 

3 

3 

1 

3 

15 

95 

3 

3 

2 

2 

16 

49 

2 

2 

1 

5 

17 

29 

1 

3 

2 

2 

18 

37 

2 

1 

1 

4 

19 

56 

2 

3 

1 

1 

20 

6 

1 

1 

2 

1 

21 

33 

1 

3 

3 

- 

22 

47 

2 

2 

1 

3 

23 

85 

3 

2 

2 

3 

24 

88 

3 

2 

3 

- 

25 

60 

2 

3 

1 

5 

26 

22 

1 

2 

3 

- 

27 

92 

3 

3 

1 

4 

28 

55 

2 

2 

3 

- 

29 

2 

1 

1 

1 

2 

30 

8 

1 

1 

2 

3 

31 

32 

1 

3 

2 

5 

32 

84 

3 

2 

2 

2 

33 

44 

2 

1 

3 

- 

34 

5 

1 

1 

1 

5 

35 

80 

3 

2 

1 

3 

36 

39 

2 

1 

2 

1 

37 

3 

1 

1 

1 

3 

38 

19 

1 

2 

2 

3 

39 

89 

3 

3 

1 

1 

40 

66 

2 

3 

3 

- 

Levels 


Slide 

Situation 

W 

■a 

R 

4> 

41 

58 

2 

3 

1 

3 

42 

24 

1 

3 

1 

2 

43 

16 

1 

2 

1 

5 

44 

79 

3 

2 

1 

2 

45 

38 

2 

1 

1 

5 

46 

77 

3 

1 

3 

- 

47 

12 

1 

2 

1 

1 

48 

59 

2 

3 

1 

4 

49 

52 

2 

2 

2 

3 

50 

30 

1 

3 

2 

3 

51 

9 

1 

1 

2 

4 

52 

78 

3 

2 

1 

1 

53 

96 

3 

3 

2 

3 

54 

72 

3 

1 

2 

1 

55 

65 

2 

3 

2 

5 

56 

86 

3 

2 

2 

4 

57 

61 

2 

3 

2 

1 

58 

67 

3 

1 

1 

1 

59 

41 

2 

1 

2 

3 

60 

73 

3 

1 

2 

2 

61 

51 

2 

2 

2 

2 

62 

13 

1 

2 

1 

2 

63 

10 

1 

1 

2 

5 

64 

69 

3 

1 

1 

3 

65 

18 

1 

2 

2 

2 

66 

99 

3 

3 

3 

- 

67 

54 

2 

2 

2 

5 

68 

28 

1 

3 

2 

1 

69 

81 

3 

2 

1 

4 

70 

34 

2 

1 

1 

1 

71 

75 

3 

1 

2 

4 

72 

26 

1 

3 

1 

4 

73 

76 

3 

1 

2 

5 

74 

62 

2 

3 

2 

2 

75 

74 

3 

1 

2 

3 

76 

90 

3 

3 

1 

2 

77 

70 

3 

1 

1 

4 

78 

27 

1 

3 

1 

5 

79 

83 

3 

2 

2 

1 

80 

57 

2 

3 

1 

2 

. 
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Exhibit  B.  16 

Summary  Text  of  Session  Plan  Item  B.  12.  a 
Instructions  for  Maximum  Threatening  Range 
During  the  portion  of  the  experiment  in  which  I asked  for  your  evaluation 
of  the  109  slides  depicting  exposure  to  enemy  weapons,  you  responded  with  num- 
bers between  zero  and  one  thousand  (0  - 1000)  to  indicate  the  threat  you  felt  in 
each  situation.  The  maximum  range  between  your  line  of  flight  (exposed)  and 
the  enemy  weapon  was  approximately  80%  of  the  stated  maximum  effective  range. 
Presumably,  you  may  have  felt  threatened  even  at  these  extreme  ranges.  The 
question  becomes,  for  each  type  of  weapon  and  for  each  enemy  weapon  bearing 
and  exposure  time,  at  what  range  do  you  no  longer  feel  threatened  by  that 
weapon.  That  is,  at  what  range  would  you  feel  free  to  enter  a zero  (0)  to  indi- 
cate the  threat  you  feel. 

Please  retrieve  the  form  you  used  earlier  during  the  experiment  to 
determine  the  situation  you  felt  to  be  the  most  threatening  of  all  situations  to  be 
presented.  Record  your  answers  on  this  form.  As  indicated  by  the  example  I 
am  going  to  present  on  the  screen,  the  range  you  record  can  change  with  bearing 
as  well  as  with  weapon  type  and  exposure  time.  You  should  record  a total  of 
forty- five  answers  in  all. 
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Exhibit  B.  18 

Summary  Text  of  Session  Plan  Item  B.  14 
Instructions  for  Unexposed  Threat  Equivalent 

You  will  note  at  this  point  that  we  have  seen  one  hundred  nine  (109)  ex- 
posure situation  slides,  and  I am  sure  each  of  you  was  relieved  when  I told  you 
we  were  finished  viewing  slides  for  awhile.  However,  there  are  still  nine  (9) 
data  cards  in  your  original  deck  that  have  not  been  used  and  I do  have  plans  for 
them. 

Earlier  I requested  that  you  analyze  a given  exposure  situation  and  re- 
cord the  threat  that  you  felt.  I am  now  going  to  show  nine  (9)  exposure  situa- 
tions that  are  identical  to  nine  (9)  of  the  situations  presented  earlier.  However, 
this  time,  instead  of  determining  threat,  I want  you  to  perform  the  following 
analysis: 

Suppose  that  you  now  have  a choice.  You  can  either  continue  along  your 
present  flight  path  and  experience  the  exposure  situation  depicted  by  the 
slide,  or  you  can  alter  your  flight  path  to  find  a new  path  that  is  com- 
pletely protected  from  the  enemy  weapon.  However,  you  can  buy  this 
protection  only  at  the  cost  of  increased  travel  time  to  your  final  objective. 
The  question  is  "By  how  much  time  would  you  be  willing  to  delay  your 
arrival  in  order  to  avoid  the  exposure  situation?"  This  is  the  value  I 
want  you  to  record  on  each  data  card  as  each  exposure  situation  slide  is 
presented.  The  answers  should  be  in  units  of  seconds. 

Now,  let  me  make  several  suggestions  to  assist  you  in  your  analysis. 
First,  please  recall  the  scenario  that  we  discussed  at  the  beginning  of  this 
session.  You  should  think  of  the  tradeoff  of  exposure  with  travel  time  in  the 
context  of  the  mission  that  was  described  as  part  of  the  scenario  description. 
Next,  if  you  feel  no  threat  in  a particular  situation  then  you  should  most 
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probably  enter  zero  (0)  seconds  as  your  answer  since  you  would  not  normally 
want  to  delay  your  arrival  to  avoid  a nonthreatening  situation.  Finally,  if  you 
feel  threatened  in  a given  situation,  you  might  start  with  a delay  time  large 
enough  to  insure  that  the  exposure  situation  would  be  preferred  to  the  delay. 
Then  you  might  decrease  the  delay  mentally  until  you  have  difficulty  deciding 
whether  exposure  or  delay  is  preferred.  At  this  point  you  will  have  the  delay 
time  that  should  be  entered  as  your  answer. 

Between  each  slide  in  the  sequence  that  I am  now  going  to  present  I will 
allow  ample  time  for  you  to  perform  your  analysis  and  record  your  answer. 
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Exhibit  B.  19 


Summary  Text  of  Session  Plan  Item  B.  16.  a 
Instructions  for  Ranking  and  Rating  Enemy  Weapons 

1.  Select  the  weapon  you  consider  to  be  the  most  threatening  overall.  Assign 
it  the  rank  of  one  (1).  Rate  it  with  a value  of  one  hundred  (100). 

2.  Select  the  next  most  threatening  weapon.  Assign  it  the  rank  of  two  (2). 
Assign  it  a rating  between  zero  and  one  hundred  (0  - 100)  based  on  its 
threat  relative  to  the  weapon  ranked  first. 

3.  Continue  until  all  weapons  are  ranked  and  rated. 

Exhibit  B.  20 

Example  of  Session  Plan  Item  B.  16.  b 

Enemy  Weapon  Ranking  and  Rating  Example 

Among  the  weapons  (14.  5 mm,  23  mm,  REDEYE)  Cpt.  Harry  Helo  feels 
the  14.  5 mm  is  the  most  threatening,  the  23  mm  85%  as  threatening  as  the  14.  5 
mm  and  the  REDEYE  40%  as  threatening  as  the  14.  5 mm.  Harry's  ranking  and 
rating  chart  for  enemy  weapons  appears  as  follows: 

Rank  Weapon  Rating 

1 1/1  C m yy*  _ lQQ 

- 8’^ 

- 50 

■ * o 

0 
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Table  C.2 

Cross-Country  and  Attack  Speeds 


* Numbers  in  these  columns  represent  values  used  in  Phase  II  Research. 


**  Numbers  in  parentheses  indicate  median  speed. 
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Table  C.4 

Unexposed  Threat  Factor 


Sample  Values 

Average 

6 

0.8743 

0.4878 

0 

0.1936 

0.2743 

0 

0.2763 

0 

0.2348 

7 

HEfli 

3.5897 

0.2710 

0 

0.3252 

0.2743 

0.5405 

0.2490 

0. 1380 

0.5986 

8 

0.4336 

0 

0.8130 

0.2195 

0 

1.5873 

0. 1372 

0.8695 

9 

1.2925 

0.4592 

0 

0.7819 

0 

5.0000 

0.4566 

0 

0.9989 

10 

rr 

0 

0 

0. 1105 

0 

0 

2.0000 

0 

0 

0.2345 

11 

ui'i'i'i'j 

5.3333 

1.3953 

1.5385 

2.5397 

0 

0 

1.5873 

0.7937 

1.5764 

12 

1.5000 

mm 

HSU 

0.1105 

0 

0 

0. 2857 

0 

0 

0.2593 

13 

0 

0.7104 

0 

0 

0.5464 

0 

0 

0.0028 

0. 1897 

0.1610 
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Table  C.5 

Maximum  Threatening  Range* 


Subject 

Weapon 

Type 

Exposure 

Time 

(seconds) 

Azimuth 

0° 

45° 

90° 

135° 

O 

O 

00 

6 

14 . 5 mm 

5 

1000 

1000 

1000 

1000 

1000 

15 

1500 

1600 

1800 

1600 

1500 

45 

2000 

2000 

2300 

2000 

2000 

23  mm 

5 

2000 

2500 

2500 

2500 

2000 

15 

3000 

3000 

3000 

3000 

3000 

45 

3000 

3500 

3500 

3500 

3000 

EraaBa 

5 

2000 

2000 

3000 

2000 

2000 

15 

3000 

3000 

4000 

4000 

4000 

45 

4000 

4000 

4500 

4500 

4500 

7 

14.5  mm 

5 

0 

800 

1200 

800 

0 

15 

B 

■B 

45 

■Ej!B 

V 

B 

wrnrn 

B B 

23  mm 

5 

0 

WM 

b 

15 

2000 

wrnm 

BT7TV 

B ^BB 

45 

3200 

3200 

3200 

3200 

IflS  B 

Redeve 

5 

n 

B 

wmu 

B B 

15 

n 

wrnrn 

wSim 

wwi 

45 

4000 

4000 

4000 

mmm 

8 

14.5  mm 

5 

0 

0 

0 

0 

0 

15 

0 

0 

0 

1400 

0 

45 

1500 

1500 

0 

1500 

1500 

23  mm 

5 

0 

0 

0 

0 

0 

15 

0 

3000 

1500 

3000 

3000 

45 

300 

3000 

2000 

3000 

3000 

Redeye 

5 

0 

0 

0 

3500 

3500 

15 

0 

0 

2000 

4000 

4000 

45 

0 

0 

3400 

5000 

5000 

* 

All  entries  in  meters . 
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Table  C. 5 


Maximum  Threatening  Range*(continued) 

Subject 

Weapon 

Type 

Exposure 

Time 

(seconds) 

Azimuth 

0° 

-o 

45 

O 

o 

<7i 

135° 

180° 

9 

14 . 5 mm 

5 

0 

0 

0 

0 

0 

15 

1000 

2000 

2000 

2000 

1000 

45 

2000 

2500 

2500 

2500 

2000 

23  mm 

5 

0 

0 

0 

0 

0 

15 

3000 

4000 

4000 

4000 

3000 

45 

4000 

4500 

4500 

4500 

4000 

5 

0 

0 

0 

0 

0 

15 

2000 

3000 

4000 

5000 

5000 

45 

4000 

5000 

5000 

5500 

6000 

10 

5 

1500 

n 

■ 

15 

RSil 

EJTIBI 

K3SIIH 

45 

1800 

1800 

1800 

1800 

1800 

23  mm 

5 

3000 

2500 

3000 

0 

3000 

15 

3000 

3500 

3500 

3500 

3000 

45 

3500 

3500 

3500 

3500 

3500 

Redeye 

5 

0 

1000 

2000 

2800 

3400 

15 

0 

1500 

2200 

3000 

3800 

45 

0 

1600 

3000 

3800 

11** 

14 . 5 mm 

5 

0 

0 

0 

0 

0 

15 

1500 

1500 

1500 

1500 

1500 

45 

2000 

2000 

2000 

2000 

2000 

23  mm 

5 

0 

0 

0 

0 

0 

15 

3000 

3000 

3000 

3000 

3000 

45 

3500 

3500 

3500 

3500 

3500 

Redeye 

5 

— 

— 

- 

- 

15 

- 

- 

- 

- 

- 

45 

- 

- 

- 

- 

- 

** 


All  entries  in  meters . 

Subject  11  failed  to  provide  Redeye  data. 
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Table  '■  i> 

Maximum  Threatening  Range*(continued) 


Subject 


12 


Weapon 

Type 

Exposure 

Time 

(seconds) 

Azimuth 

0° 

ArO 

45 

CD 

o 

o 

135° 

*— » 
oo 
o 
c 

14.5  mm 

5 

1200 

1200 

1400 

1300 

1400 

15 

1375 

1350 

1200 

1400 

1300 

45 

300 

600 

700 

500 

400 

23  mm 

5 

2500 

2800 

2700 

2900 

2800 

15 

2000 

2200 

2100 

2200 

2300 

45 

500 

500 

600 

700 

600 

Redeye 

5 

3000 

2500 

3000 

2500 

2500 

15 

2000 

3000 

3000 

3000 

2500 

45 

1500 

1500 

2400 

1200 

1000 

14 . 5 mm 

5 

0 

0 

0 

0 

0 

15 

1250 

1250 

1250 

1250 

1250 

45 

1400 

1400 

1400 

1400 

1400 

23  mm 

5 

0 

o 

0 

0 

0 

15 

2700 

2700 

2700 

2700 

2700 

45 

3500 

3500 

3500 

3500 

5 

0 j 

0 1 

0 

0 

0 

15 

in 

i wm 

0 

4000 

4000 

45 

— 

■T7JTITB 

mrnrn 

mmm 

* All  entries  in  meters. 
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Table  C.6 

Threat  Values  for  14.5  mm  Weapon 


Range  (meters) 


Exposure 

0 

600 

Time 

■ 

Azimuth 

Azimuth 

Subject 

(seconds) 

1 

0° 

45° 

90° 

135° 

180° 

0° 

, -O 
4n 

90° 

135° 

180° 

6 

5 

0 

3 

0 

4 

0 

0 

100 

1 

3 

0 

0 

15 

25 

700 

800 

500 

50 

300 

700 

45 

850 

900 

800 

400 

600 

900 

700 

7 

5 

■n 

0 

0 

0 

0 

100 

50 

0 

0 

0 

15 

200 

51 

400 

700 

200 

300 

100 

300 

300 

mm 

45 

400 

400 

500 

800 

400 

800 

900 

500 

8 

5 

0 

m 

0 

200 

0 

0 

200 

100 

100 

100 

0 

15 

200 

ETiTil 

300 

200 

750 

600 

200 

400 

400 

1000 

UjjJI 

45 

700 

cm 

600 

800 

1000 

600 

800 

500 

750 

1000 

■sen 

9 

5 

0 

45 

0 

5 

0 

50 

0 

0, 

'll 

0 

0] 

15 

120 

950 

960 

950 

775 

600 

200 

250 

wm 

45 

999 

999 

985 

997 

925 

795 

850 

700 

987 

890 

10 

5 

0 

0 

100 

0 

0 

0 

50 

0 

0 

0 

100 

15 

700 

cm 

800 

600 

900 

900 

800 

900 

800 

400 

400 

45 

1000 

TtTiTfl 

750 

1000 

950 

700 

1000 

900 

900 

11 

5 

0 

0 

100 

50 

0 

0 

0 

0 

50 

200 

200 

15 

100 

mt 

400 

400 

700 

850 

300 

300 

900 

200 

400 

45 

700 

SSI 

800 

900 

700 

900 

300 

800 

850 

900 

12 

5 

200 

o 

o 

10 

100 

300 

300 

10 

m 

10 

15 

990 

EE23 

700 

400 

800 

800 

400 

600 

800 

200 

700 

45 

1000 

cm 

900 

980 

1000 

990 

970 

980 

13 

5 

0 

0 

o 

0 

0 

0 

0 

0 

0 

0 

0 

15 

50 

450 

350 

650 

750 

650 

650 

700 

750 

300 

650 

45 

900 

800] 

900 

975 

900 

_750~1 

900 

850 

~99!r 
. ■ — 

8001 

950 
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Exposure 

Time 

Subject  (seconds) 


Range  (meters 


1200 


Azimuth 


0°  45°  90° 


2400 


Azimuth 


90°  135C 


01  100 


880  I 700  I 500  I 800 


1000  1000  850 


0 0 100 


500  200  600  800 


800  0000  1000  500 


0 100  0 


3501200  500  1000  700 


600  1 800  1000  800 


0 0 0 


IBS! 

\m\ 


01  0 


HI 


0 0 


750  920 


975  999  1999  I 999  I 975 


100  100  100  100  100 


100(1 900  1000  700  1000 


lOOplOOO  1000  1000  900 


1 100  100  100  100  200 


600  200  800  700 


1950950  1000  1000  950 


500  100  100  100  100 


800  850  890  900  980 


1000  990  1000  800  1000 


0 0 0 0 0 


400  550  500  150  550 


|975  |975  995  980  950 


K 


56 


890  999  999 


0 0 100 


400  200  600 


4001 500  600  1000 


100  100  I 300 


200  600 


800 


800  800 


300  50 


850  600 


950 


0 0 0 0 


450  500  450  250 


750  900  975  780 


500 

900 

600 

0 

850 

700 

1000 

800 

0 

0 

500 

250 

940 

950 

0 

100 

1000 

800 

1000 

800 

I 

I 

I 


Table  C.8 

Threat  Values  for  Redeye  Weapon 


Ranr;e  (meters) 


Subject 

Exposure 

Time 

(seconds) 

l_0 

1400 

2800 

i 

Azimuth 

Azimuth 

0° 

.cO 

45 

90° 

135° 

180° 

0° 

4o 

90° 

135° 

180° 

6 

5 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

15 

800 

tm 

900 

999 

1000 

1000 

900 

990 

999 

1000 

1000 

45 

FTTiffl 

im 

1000 

1000 

1000 

(1000 

900 

1000 

1000 

1000 

1000 

7 

5 

0 

L~o 

100 

100 

100 

1 100 

0 

0 

0 

15 

900 

fjTTff! 

900 

900 

750 

900 

500 

I 

45 

1000 

ETi!5 

1000 

1000 

JTT51 

1000 

— 

1000 

1000 

EH 

IPT51 

8 

5 

0 

0 

0 

0 

100 

100 

01 

0 

0 

■n 

■n 

15 

200 

150 

200 

500 

750 

600 

0 

0 

200 

800 

800 

45 

600 

E33 

600 

500 

1000 

1000 

300 

200 

500 

1000 

1000 

9 

5 

0 

0 

0 

25 

0 

100 

0 

0 

0 

0 

10 

15 

875 

300 

50 

ninth 

955 

940 

15 

0 

na 

900 

995 

45 

1000 

450 

700 

999 

999 

25 

910 

100  | 

999 

999 

10 

5 

0 

L o 

200 

0 

100 

100 

15 

800 

400 

0 

800 

900 

45 

1000 

200 

100 

TiliTiM 

Si 

1000 

0 

1000 

EE 

1 1000 

11 

5 

0 

0 

100 

0 

PiBB 

1000 

m i 

sni 

JfJJ 

E 

15 

200 

0 

150 

500 

900 

0 

i 

B5I 

45 

EH 

200 

100 

750 

950 

1000 

0 

100 

1551 

1551 

12 

5 

5 

50 

10 

10 

400 

10 

50 

5 

wm 

wm 

15 

950 

H51 

900 

1551 

in 

BPfSI 

w 

45 

1000 

900 

200 

950 

900 

1000 

900 

1551 

BW1 

13 

5 

0 

0 

0 

0 

0 

50 

0 

0 

flEfl 

wm 

w 

15 

90 

0 

0 

350 

250 

0 

0 

0 

525 

1351 

45 

990 

350 

350 

350 

850 

990 

10 

950 

990 
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Exhibit  D.l 


Experimental  Session  Plan 
Part  A Phase  II  Route  Selection  Session 
A.l  Present  introduction 

A.  2. a Pass  out  TOW  Weapon  System  Description 

A. 2.b  Present  discussion  of  TOW  Description 

A.2.c  Pause  to  permit  subjects  to  review  TOW  Description 

A . 3.  a Pass  out  Map  Boards 

A.3.b  Present  discussion  of  Map  Boards 

A.3.c  Pause  to  permit  subjects  to  familiarize  themselves  with 
Map  Boards 

A. 4 Present  discussion  of  miscellaneous  route  preparation  materials 

A. 5. a Pass  out  route  preparation  overlay  and  Route  Description  Form 
for  first  route 

A.5.b  Present  instructions  for  using  Route  Description  Form  and  for 
preparing  route  sketch 

A.S.c  Pause  to  permit  subjects  to  complete  first  route  description 

A. 6 Repeat  steps  A. 5. a and  A.5.c  for  routes  two,  three,  and  four. 
Let  each  subject  work  at  his  own  pace 

A.  7 Collect  all  material  from  each  subject 

Part  B Phase  II  Route  Evaluation  Session 

B. l  Present  introduction 

B.2  Pass  out  supply  package 

B.3.a  Present  instructions  for  transferring  routes 


I 
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B.3.b  Pause  to  permit  subjects  to  transfer  routes 


B.4.a  Present  instructions  for  the  use  of  computer  prepared  route 
overlays  and  answer  sheets  in  performing  route  evaluations 

B.4.b  Pause  to  permit  subjects  to  perform  route  evaluations 

B.5  Collect  all  material  from  each  subject 


I 

I 

I 
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Exhibit  D.  2 


Summary  Text  of  Session  Plan  Item  A.  1 
Introduction  to  Route  Selection  Session 

Gentlemen,  we  now  come  to  the  part  of  the  experiment  which  you  will 
probably  find  to  be  the  most  interesting  of  all.  The  slides,  which  I know  you've 
grown  a little  tired  of,  are  all  behind  us  now.  For  the  rest  of  the  session  we 
will  be  concentrating  in  an  area  that  will  permit  you  to  express  your  own  style — 
to  demonstrate  the  creativity  that  makes  you  a successful  military  aviator.  I 
am  going  to  present  you  with  a series  of  realistic  tactical  situations  and  ask  you 
to  prepare  a sketch  of  a route  you  would  follow  while  executing  a mission  in 
response  to  each  of  the  situations. 

We  will  start  by  providing  you  with  detailed  information  about  the  arma- 
ment system  that  will  be  available  to  you  during  the  missions.  We  will  then  pro- 
vide each  of  you  with  a map  board  that  will  illustrate  not  only  the  terrain  and 
vegetation  in  the  mission  operations  area  but  also  the  overall  tactical  situation 
which  you  will  face.  We  will  also  provide  a variety  of  materials,  such  as  scales 
and  marking  pens,  which  will  be  of  use  in  preparing  your  route  sketches. 

Finally,  a series  of  four  route  selection  situations  will  be  defined  for  each  of 
you,  from  which  four  different  route  sketches  may  be  prepared.  Blank  acetate 
overlays  will  be  provided  for  the  route  sketches,  and  answer  sheets  will  be  made 
available  to  record  certain  data  about  each  route. 

Now,  as  we  commence  the  exercise  let  me  remind  you  of  several  items 
which  are  very  important  to  the  success  of  the  experiment.  First,  as  I have 
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said  before,  there  are  no  right  or  wrong  answers.  We  want  only  your  opinion 
as  a highly  qualified  military  aviator.  Your  skill  as  a pilot  is  not  in  question. 
Second,  I want  you  to  recall  the  scenario  I outlined  for  you  at  the  beginning  of 
the  session.  As  you  prepare  your  route  sketches  please  remember  not  only  that 
the  missions  are  being  conducted  in  response  to  particular  situations  but  also 
that  the  scenario  provides  a frame  of  reference  within  which  all  the  situations 
arise.  We  are  interested  in  your  actions  in  a hypothetical  mid-intensity  warfare 
environment;  one  in  which  you  are  providing  direct  fire  support  for  an  armored 
unit  engaged  with  a highly  lethal  enemy. 


i 
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Exhibit  D.  3 


Example  of  Session  Plan  Item  A.  2.  a 


Section  IV.  XM26  (TOW)  ARMAMENT  SUBSYSTEM 


FM  l-«0 


|-9.  Description 

The  purpose  of  the  XJI26  (TOW)  helicopter  ar- 
mament subsystem  (fig.  1-2)  is  to  provide  the 
Army  with  a highly  mobile,  airborne,  heavy  point 
lire  weapon  system.  This  system  uses  the  TOW 
(tube  launched,  optically  tracked,  wire  command 
link)  guided  missile.  The  XM26  is  designed  to  re- 
place the  M22  subsystem.  It  consists  of  the  fol- 
lowing major  assemblies — 


a.  Stabilized  Sight/Sensor.  The  stabilized 
sight/sensor  (A,  fig.  1-2)  mounted  in  the  helicop- 
ter nose  is  the  key  to  the  XM26  subsystem.  This 
gyro-stabilized  sight  enables  the  gunner  to  keep 
the  crosshairs  of  his  sight  on  the  target,  regard- 
less of  helicopter  vibration  and  maneuvers. 
Through  the  sight,  the  gunner  establishes  and 
maintains  the  line  of  sight  which  the  missile  fol- 
lows to  the  target. 


b.  Launcher*.  The  missile  container  is  attached 
to  the  helicopter  bomb  rack.  There  is  a launcher 
containing  three  missiles  (B,  fig.  1-2)  on  each 
side  o:  the  UH-1  helicopter.  The  launchers  are 
trainable  in  elevation  only  by  hydraulic  actua- 
tion. 

c.  Electronics.  The  electronic  assemblies  con- 
t..ir.  necessary  circuitry  for  the  operation  of  all 
XM26  subsystem  assemblies.  The  circuitry  is 
built  into  three  separate  chassis — rervo  electron- 
ii auxiliary  electronics,  and  commend  signal 
generator 


d.  Power  Supply.  Power  for  the  subsystem  is 
produced  through  an  inverter  located  in  the  heli- 
copter engine  compartment.  The  inverter  uses  21 
volts  DC  from  the  helicopter  emergency  power 
and  converts  it  into  llo  volts  AC  100  cycle, 
three-phase  power  for  the  XM26  subsystem.  If 
emergency  power  is  need  ni  as  helicopter  prime 
power,  the  circuit  is  designed  so  that  the  X.M26 
subsystem  can  be  switched  out  of  the  circuit. 

c.  Missile.  The  TOW  missile  round  is  launched 
directly  from  the  missile  container  (fig.  1— S) , 
without  an  .idiIition.nl  launch  tube. 
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Figure  1-3.  Missile  and  container. 


/.  Cockpit  Controls  and  Displays.  The  cockpit 
controls  and  displays  located  in  the  cockpit  allow 
for  ease  of  operation  by  the  (runner  and  do  not  in- 
terfere with  the  flipht  controls  of  the  helicopter. 


lit! 


Figure  1-4  shows  cockpit  controls  and  displays  in 
a UH-1  helicopter;  however,  the  XM26  subsys- 
tem is  adaptable  to  the  AH-1  attack  helicopter. 
Power  and  armament  controls  are  mounted  on 
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Figure  J-l.  Cockpit  control*  aim'  dx-plngr  in  UH-1  helicopt:.-. 
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the  consol?  between  the  pilot  and  gunner’s  sent, 
'.'lies?  are  wilh!r.  easy  reach  of  both  pilot  and 
gunner.  Additional  controls  and  displays  are  lo- 
cated on  the  sight  extension  tube,  gunner’s  arm- 
rest, and  the  helicopter  instrument  panel. 

(1)  Pilot  steering  display.  The  pilot  steering 
display  (fig.  1-5)  provides  attack  status  data  and 
an  azimuth  steering  signal  for  missile  firing.  So 
that  the  pitot  will  not  exceed  the  gin.bal  angle 
limits  during  maneuvering  flight,  it  indicates 
platform  gimbal  ar.gie3. 


Figure  1-5.  Pilot  steering  display. 


(2)  TOW  armament  subsystem  control 
panel.  The  TOW  armament  subsystem  control 
panel  (fig.  1-6)  has — 


Mvsioor 

Figure  IS.  TOW  arnutruiit  eubei/etem  control 
panel. 


(a)  SAFE/ARMED  switch.  When  this 
switch  is  placed  in  the  SAFE  position,  a missile 
cannot  be  fired.  When  placed  in  the  ARMED  posi- 
tion, this  switch  completes  the  circuitry  neces- 
sary for  firing  a missile. 

(b)  MISSILE  SELECT  switch.  Missile 
selection  can  be  manual  or  automatic  by  means  of 
the  MISSILE  SELECT  switch.  When  placed  in 
the  AUTO  position,  the  system  will  select  the 
next  ready  missile.  Each  position  on  the  switch 
has  a flag  indicator  that  will  indicate  the  pres- 
ence (MSL),  or  absence  (barber  pole)  of  the  se- 
lected missile. 

(c)  JETTISOM  switches.  Ir.  an  emer- 
gency, activation  of  the  JETTISON'  switches  jet- 
tisons the  launcher  assemblies  to  improve  heli- 
copter maneuverability. 

( d ) EMEItgency  WIRE  CUT.  In  case  of 
wire  entanglement  or  failure  of  the  automatic 
wire  cutting  device,  the  emergency  wire  cut  con- 
trol permits  the  gunner  to  cut  the  wire. 

(3)  TOW  weapon  subsystem  control  panel. 
The  TOW  weapon  subsystem  control  panel  (fig. 
1-7)  has — 


Figure  1-7.  TOtt'  weapon  tubsyetem  control  panel. 


la)  POWER  OS /OFF  switch.  In  the  ON 
position,  full  operating  power  is  applied  to  the 
subsystem. 

(Ii)  SYSTEM  TEST  switch.  This  10-posi- 
tion switch  is  used  to  determine  the  operational 
status  of  the  weapon  subsystem.  Tile  first  posi- 
tion (OPERATE)  switches  out  all  system  test 
circuitry;  the  switch  must  be  in  this  position  to 
fire  a missile.  The  second  position  (LAMP  Test) 
applies  power  to  all  indicator  lamps.  The  next 
two  positions  (El  and  Bi)  are  used  in  perform- 
ing boresighting  checks  jr.d  alir.emert.  The  last 
six  positions  of  the  switch  are  used  to  perform 
system  selftest.  Results  of  the  selftcst  are  indi- 
cated on  the  three  flag  indicators  above  the 
switch.  A CO  indication  will  be  displayed  wr.on  a 
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particular  test  has  been  satisfactorily  completed. 
The  throe  indicators  used  are — SIGHT,  for  the 
stabilised  sight/ sensor  ur.it;  ST  A it , lor  stabilisa- 
tion electronics;  and  CSG,  for  missile  guidance 
electronics. 

(•1)  Track  control  unit.  The  track  control 
unit  (FIG.  1-8)  is  designed  for  right-handed  op- 
eration by  the  gunner  to  provide  control  inputs  to 
the  TOW  sight  system  for  manual  acquisition, 
tracking,  and  stow  positioning  cf  the  sight.  The 
tracking  control  stick  and  control  switch  have 
been  designed  into  a small  uni:  separate  from  the 
armrest.  The  position  of  the  control  unit  may  be 
adjusted  to  accommodate  different  sized  gunners. 


(5)  Gunner's  handi/rip.  The  gunner's  hand- 
grip (fig.  I-!))  provides  a means  of  stabiliza- 
tion for  the  gunner,  switching  function  controls, 
and  a designation  pointer  display.  The  handgrip 
is  fixed  to  the  left  sice  of  the  sight  unit  relay  col- 
umn near  the  upper  rotary  joint.  The  switching 
functions  are  short  or  long  range  selection,  high 
or  low  magnification  selection,  camera  control,  re- 
ticle brightness  control,  attack  mode  selection, 
ar.d  trigger.  The  trigger  is  a recessed,  momen- 
tary, snap-action  pushbutton.  The  designation 
pointer  display  is  a small  pencil-shaped  pointer 
that  indicates  where  the  sight  is  pointing.  It  is 
used  in  the  heads-up  mode  for  gross  positioning  of 
the  sight  prior  to  headsdowr.  acquisition.  It  is  lo- 
cated over  a moving  card  display  that  indicates 
the  azimuth  position  of  the  sight  relative  to  a 
fixed  scale.  > 

1-10.  Operation 

a.  In  combat,  the  helicopter  gunner  acquires 
and  tr..rks  a target  through  the  stabilized 
s'ght  sen.-, or.  A display  on  the  mscru.T.snr  panel 
(fig.  1— .")  will  indicate  to  the  pilot  each  phase  of 
the  engagement ; c.g.,  attack,  tire,  and  maneuver. 
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Figure  l~9.  Handgrip. 

Also  indicated  are  limitations  of  the  evasive  ma- 
neuvers that  the  piiot  may  perform. 

t.  Easily  operated,  viscous  damped  controls 
(fig-.  I-S)  enable  the  gunner  to  maintain  the 
sight/sensor  pointing  direction  without  regard  to 
the  angular  motion  or  vibration  of  the  helicopter. 

c.  The  TOW  missile  is  fired  ir.ta  the  field  of 
view  of  the  stabilized  sight/ser.sor.  The  stabilized 
sight. /.sensor  "senses"  missile  fiightpath  deviation 
from  the  line  of  sight  maintained  by  the  gunner. 
These  deviations  art  sent  (in  terms  of  electronic 
impulses)  to  the  command  signal  generator,  then 
converted  into  corrective  commands  and  sent  to 
the  missile  by  thin  wires  that  arc  payed  out  in 
flight.  These  coireetivc  e.  uimands  puide  the.  mis- 
sile back  to  the  line  of  sight  and  on  io  the  target 
(fig.  1-10). 

1-11.  Logistical  Support 

TOW  armament  subsystem  equipment  will  be  de- 
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F igure  1-10.  Weapon  concept. 


signed  tc  include  a completely  built-in  selftest  ca- 
pability '.hat  will  isolate  faults  down  to  a remov- 
able major  assembly.  Under  the  current  planned 


concept,  the  aircraft  armament  repairman  will 
remove  and  replace  the  defective  assemblv  indi- 
cated by  the  built-in  solftest. 


\ 


ACw  rjUiA 


1-7 


366 


Auxiliary  Data 


XM26(TOW)  Armament  Subsystem 
(Unclassified — for  illustrative  purposes  only) 

Time  required  to  fire  a round  after  the  aircraft  has  achieved  its  launch 
heading — 5 seconds. 

Maximum  aircraft  heading  deviation  from  line  of  flight  of  missile  after 
launch — 40  degrees. 
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Maximum  Range  2500  m 
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Exhibit  D.  4 
Summary  Text  of  Session  Plan  Item  A.  2.  b 
Discussion  of  TOW  Weapon  System 

If  you  will  examine  your  supply  box,  you  will  find  a brochure  describing 
the  armament  system  we  want  you  to  visualize  as  being  your  sole  source  of  fire- 
power while  executing  the  four  missions  that  we  are  going  to  define.  The  sys- 
tem is  the  XM26,  or  TOW,  armament  subsystem. 

The  first  five  pages  of  the  brochure  present  a general  discussion  that 
defines  the  various  components  of  the  system  and  describes  the  way  in  which  the 
system  may  be  employed.  The  last  page  presents  some  detailed  operational 
data  which  you  will  have  need  of  when  you  prepare  your  route  sketches.  Please 
note  that  none  of  the  material  is  classified. 

Now,  as  you  examine  the  material  in  the  brochure  let  me  emphasize 
two  points  that  are  very  important  which  you  should  remember  for  the  remainder 


of  the  session.  First,  as  you  prepare  your  route  sketches  you  must  insure  that 
they  include  an  attack  that  does  not  violate  the  operational  limitations  of  the 
armament  system.  This  means  that  the  launch  range  must  be  within  the  mini- 
mum and  maximum  ranges  specified  for  the  missile.  Furthermore,  you  must 
have  the  target  in  sight  for  a sufficient  time  prior  to  launch  to  permit  accurate 
aiming.  Finally,  while  the  missile  is  flying  you  must  continue  to  guide  it.  This 


means  that  you  must  maintain  visual  contact  with  the  target  and  that  the  gimbal 
limits  of  the  tracking  mechanism  limit  the  amount  by  which  your  flight  path  may 
deviate  from  that  of  the  missile. 


Second,  the  operational  data  to  be  used  in  defining  the  attack  limits  I 
have  just  discussed  are  those  that  appear  in  the  weapon  system  brochure.  How- 
ever, if  you  are  already  familiar  with  the  TOW,  you  will  note  that  the  opera- 
tional data  in  the  brochure  are  not  exactly  those  with  which  you  are  familiar. 

I have  revised  the  correct  data  slightly  in  order  to  keep  this  experiment  un- 
classified. Thus,  those  of  you  who  are  familiar  with  the  system  must  be 
especially  careful  to  use  the  brochure  data.  Otherwise,  we  might  obtain  route 
sketches  which  are  unacceptable  as  far  as  the  experiment  is  concerned. 
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Exhibit  D.  5 


Description  of  Map  Boards  Used  in  Session  Plan  Item  A.  3 

As  explained  in  Chapter  4 and  Appendix  A,  a total  of  four  unique  map 
boards  were  required  for  the  Phase  II  Experiment  Route  Selection  Session.  Map 
board  A was  used  by  subjects  6 and  10,  map  board  B by  subjects  7 and  11,  and 
so  on.  Each  map  board  was  constructed  of  the  following  materials: 

1.  One  (1)  piece  of  heavy  white  poster  board,  measuring  20  inches 
by  12  inches; 

2.  One  (1)  battlefield  contour  map  on  white  paper  stock,  measuring 
14  1/2  inches  by  7 1/2  inches; 

3.  One  (1)  forest  overlay  on  clear  acetate,  measuring  14  1/2  inches 
by  7 1/2  inches; 

4.  Two  (2)  line  of  sight  overlays  on  clear  acetate,  each  measuring 
14  1/2  inches  by  7 1/2  inches;  and 

5.  One  (1)  protective  sheet  of  clear  acetate  measuring  18  inches 
by  10  inches. 

The  quoted  dimensions  of  the  contour  map  and  overlays  are  the  dimensions  of 
the  boundaries  of  the  battlefield,  and  examples  of  these  materials  may  be  found 
in  Appendix  A.  A representation  of  the  complete  map  board  may  be  found  at 
Exhibit  D.  6.  , 

The  reader  may  note  by  examining  Exhibit  D.  6 that  the  finished  map 
boards  were  potentially  very  difficult  to  interpret.  However,  a coloring  scheme 
was  used  that  alleviated  this  difficulty.  Unfortunately,  the  coloring  scheme  is 
not  apparent  in  the  exhibit. 
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As  presented  to  the  subjects,  the  forested  regions  were  outlined  in 
green,  the  areas  intervisible  with  enemy  weapon  number  one  were  outlined  in 
red  and  the  areas  intervisible  with  enemy  weapon  number  two  were  outlined  in 
blue.  It  was  found  that  after  sufficient  exposure  to  the  map  boards  and  instruc- 
tions for  interpreting  them  the  subjects  had  no  difficulty  in  visualizing  the 
situation  described  by  the  map  board. 

The  acetate  overlays  used  in  constructing  the  map  boards  were  pie- 
pared  from  plots  of  forested  regions  and  intervisibility  areas  originally  pro- 
duced by  the  computerized  methods  described  in  Appendix  A.  The  plots,  orig- 
inally on  a translucent  material,  were  reproduced  on  special  8 1/2  x 11  acetate 
material  using  the  Xerox  process.  Since  the  plots  were  oversized,  each  over- 
lay actually  consisted  of  two  acetate  sheets,  which  together  covered  the  entire 
battlefield.  Also,  since  the  Xerox  process  normally  produces  slightly  reduced 
copy,  special  adjustments  were  made  to  the  machine  that  was  used  in  order  to 
obtain  exact  copies. 


371 


AD-A045  094  OHIO  STATE  UNIV  COLUMBUS  SYSTEMS  RESEARCH  GROUP  F/G  1/2 

THE  HELICOPTER  ROUTE  SELECTION  MODEL  (DYNFLITE) . (U) 

AUG  76  G M CLARK » D C HUTCHERSON*  D C BITTERS  DAAG25-70-C-0311 
UNCLASSIFIED  RF-297B-FR-76-4 ( 0 ) NL 

5 of 6 ■■  IBHIHHiBiBil 


AD 

A045094 


Exhibit  D.  7 


Summary  Text  of  Session  Plan  Item  A.  3.  b 
Discussion  of  Map  Boards 

Now  that  you  have  become  familiar  with  the  armament  you  will  be  using 
on  your  missions,  let  us  find  out  where  the  missions  are  going  to  be  flown.  I 
am  handing  each  of  you  a map  board  that  will  do  this  job. 

You  will  note  that  there  is  a lot  of  information  on  your  map  board;  and, 
at  first  glance,  it  appears  quite  complex.  However,  after  you  have  had  a chance 
to  become  familiar  with  it,  you  will  find  that  the  information  presented  is  very 
valuable  to  you  in  your  route  selection  task. 

First,  you  will  note  that  your  map  is  of  an  area  that  is  five  kilometers 
wide  and  ten  kilometers  long,  located  in  the  Fulda  Gap  region  of  West  Germany. 
The  contour  plot  was  prepared  by  computer  using  actual  elevation  data  provided 
by  the  Army  Map  Service.  It  not  only  shows  contour  lines  at  five  meter  eleva- 
tion intervals,  but  it  also  indicates  the  elevation  of  specific  points  located  at 
regular  intervals  throughout  the  area. 

Grid  coordinates  of  any  point  can  be  determined  from  the  scales  appear- 
ing along  each  boundary  of  the  map.  The  major  division  of  these  scales  is  the 
kilometer,  but  each  kilometer  is  also  divided  into  five  parts  so  that  a scale 
reading  is  available  at  intervals  of  two- tenths  of  a kilometer.  For  ease  of 
orientation  let  us  assume  that  north  is  to  the  top  and  east  is  to  the  right  as  is 
customary. 
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Now,  the  map  boards  also  indicate  regions  of  the  battlefield  that  are 


covered  with  woods  and  forests.  These  regions  are  outlined  in  green  and  are 
the  same  for  each  one  of  you.  Trees  within  these  forested  regions  average 
twenty  meters  in  height  while  outside  the  forested  regions  the  terrain  is  covered 
only  with  grass  and  scattered  bushes. 

Finally,  you  will  note  that  two  enemy  weapon  positions  are  indicated  on 
your  map  board  and  are  labeled  "1"  and  "2".  Each  of  you  has  a different  set  of 
weapon  positions. 

Associated  with  each  of  the  enemy  positions  is  an  overlay  that  we  will 
call  a line  of  sight  or  LOS  map.  These  LOS  maps  show  up  on  your  map  boards 
as  groups  of  irregularly  shaped  areas  outlined  either  in  blue  or  in  red.  The 
red  LOS  map  is  associated  with  the  enemy  position  labeled  with  the  red  numeral 
while  the  blue  LOS  map  is  associated  with  the  position  labeled  with  the  blue 
numeral. 

Now,  what  do  the  two  LOS  maps  reveal  about  the  battlefield?  Suppose 
you  select  any  position  on  the  battlefield  and  imagine  that  you  are  flying  a knap 
of  the  earth  flight  profile.  This  means  that  you  are  trying  to  fly  as  low  as  pos- 
sible at  each  point  along  your  flight  path,  climbing  enough  to  just  clear  the  tops 
of  trees  while  over  forested  regions  and  dipping  even  below  tree  top  altitude 
while  over  open  terrain.  The  LOS  maps  indicate  how  much  of  your  flight  profile 
will  be  exposed  to  the  two  enemy  weapons.  For  example,  suppose  you  select  a 
position  that  is  within  one  of  the  blue  LOS  areas.  This  means  that  your  flight 
path  at  (his  poinl  is  exposed  to  the  enemy  weapon  labeled  with  the  blue  numeral. 
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Not  only  can  he  detect  you  at  this  point  but  he  may  also  engage  you  with  direct 
fire.  Of  course,  you  are  also  provided  with  these  same  opportunities. 

Continuing,  suppose  you  are  within  a red  area.  Then  you  are  exposed 
to  the  weapon  labeled  with  the  red  numeral  and  he  may  detect  and  fire  at  you  if 
he  has  sufficient  time.  Finally,  if  your  position  is  both  within  a red  and  a blue 
area,  you  are  exposed  to  each  of  the  enemy  weapons.  Similarly,  if  your  posi- 
tion is  within  none  of  the  red  and  blue  areas,  your  position  is  completely  con- 
cealed. 

I am  going  to  stop  now  and  let  you  examine  your  map  board  until  you 
are  thoroughly  familiar  with  it.  If  you  have  any  questions  of  interpretation, 
please  let  me  know.  Also,  feel  free  to  examine  the  perspective  drawing  of  the 
battlefield  that  is  mounted  on  the  wall  to  your  left.  This  drawing  was  prepared 
by  computer  and  presents  the  view  you  would  have  at  3000  feet  if  you  were 
approaching  from  the  northwest  and  were  still  two  kilometers  from  the  nearest 
comer  of  the  battlefield.  Forested  regions  are  indicated  in  green  on  the 
drawing. 
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Exhibit  D.  9 

Summary  Text  of  Session  Plan  Item  A.  4 
Discussion  of  Miscellaneous  Route  Preparation  Materials 

Now  that  you  have  examined  your  map  boards  we  are  almost  ready  to 
begin  the  route  selection  exercise.  However,  before  we  start  let  me  draw  your 
attention  to  some  materials  you  will  need  in  order  to  do  the  job  correctly. 

In  your  supply  box  you  will  find  two  acetate  scales,  one  being  a fairly 
large  scale  and  the  other  one  being  rather  small.  Also,  clipped  to  the  back  of 
the  Enemy  Weapon  Brochure  we  used  earlier,  you  will  find  a launch  perfor- 
mance envelope  for  the  Redeye  missile.  Please  examine  these  materials  as  I 
discuss  them. 

The  large  acetate  scale  is  drawn  exactly  to  the  scale  of  your  map  board. 
Each  ring  represents  500  meters,  and  there  are  tic  marks  at  100  meter  inter- 
vals along  the  straight  line  splitting  the  rings.  This  scale  can  be  used  anytime 
you  need  to  know  the  distance  from  one  point  on  the  battlefield  to  another.  In 
particular,  you  should  use  this  scale  to  determine  a firing  point  that  is  within 
the  effective  range  of  the  TOW  missile  you  are  employing. 

The  small  acetate  scale  contains  seven  time  and  distance  charts,  one 
for  each  of  seven  flight  speeds.  Again,  the  scale  is  drawn  exactly  to  the  scale 
of  your  map  board  and  indicates  the  distance  you  would  travel  at  a particular 
flight  speed  in  a given  amount  of  time.  This  scale  is  to  be  used  in  conjunction 
with  the  missile  flight  time  graph  that  forms  the  last  page  of  the  TOW  weapon 
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system  description  you  exam  ined  earlier.  For  example,  suppose  you  elect  to 
fire  your  TOW  missile  from  a range  of  1500  meters  while  flying  at  100  knots. 
From  the  flight  time  graph  you  find  that  the  missile  requires  seven  seconds  to 
impact  from  this  range.  Then  from  the  time  and  distance  scale  for  100  knots, 
you  find  that  you  will  travel  approximately  300  meters  during  this  time.  Con- 
sequently, you  will  be  1200  meters  from  the  target  when  the  missile  impacts  if 
you  continue  to  fly  toward  the  target  during  missile  flight.  Data  such  as  these 
are  needed  in  constructing  accurate  sketches  of  routes  that  you  select. 

Finally,  the  Redeye  launch  performance  envelope  that  we  have  already 
discussed  is  also  drawn  to  the  scale  of  your  map  board.  If  you  place  this 
envelope  over  one  of  the  enemy  positions  and  orient  it  in  a given  direction,  you 
will  be  able  to  determine  the  areas  in  the  vicinity  of  the  weapon  in  which  effective 
fire  can  be  delivered  against  a helicopter  approaching  from  the  indicated  direc- 
tion. This  information  may  be  valuable  to  you  in  judging  various  route  alterna- 
tives in  the  vicinity  of  a Redeye  weapon. 


Exhibit  D.  10 


Summary  of  Route  Selection  Situations  Presented 
in  Session  Plan  Items  A.  5 and  A.  6 


Subject 

Route 

Map  Board 

Description 

Starting  Coordinates *  **| 

Pair 

Situation 

Weapon  1 

Weapon  2 

X 

Y 

6,10 

1 

A 

*14.  5 mm 

23  mm 

0 

4000 

2 

*14.  5 mm 

23  mm 

6000 

5000 

3 

14.  5 mm 

*23  mm 

0 

4000 

4 

14.  5 mm 

*23  mm 

6000 

5000 

7,11 

1 

B 

23  mm 

*14.  5 mm 

0 

2 

23  mm 

*14.  5 mm 

5500 

3 

*23  mm 

14.  5 mm 

0 

2000 

4 

*23  mm 

14.  5 mm 

5500 

5000 

8, 12 

1 

C 

*14.  5 mm 

23  mm 

0 

2000 

2 

*14.  5 mm 

23  mm 

5500 

5000 

3 

14.  5 mm 

*23  mm 

0 

2000 

4 

14.  5 mm 

*23  mm 

5500 

5000 

d,  13 

1 

D 

*23  mm 

Redeye 

0 

B 

2 

*23  mm 

Redeye 

5500 

3 

Redeye 

*23  mm 

0 

2000 

4 

Redeye 

*23  mm 

5500 

5000 

* Weapon  to  be  attacked 

**  Terminal  coordinates  of  all  routes  is  the  X - 0 map  boundary. 
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Exhibit  D.  11 


Example  of  Session  Plan  Item  A.  5.  a 

Map  Board  B 

ROUTE  DESCRIPTION  FORM 

I.  Subject  Number 'J_ Route  Number j_ 

II.  Starting  Coordinates:  X = O Y = £ 00 C 

III.  Terminal  Boundary:  x 

IV.  Weapon  Descriptions 

Type  X Y 

J?3  /msm;  ? 7 £>  J?SOO 

2 / £ /rr’srrU  % / O o Q P & 

V.  Weapon  to  be  Attacked  . " 


VL  Attack  Description 

A.  Mode  of  Attack  (check  one) 

1.  Hover  Fire 

2.  Running  Fire 

B.  Approximate  launch  point  coordinates  (meters) 

X = Y = 

C.  Approximate  launch  range  (meters) 

D.  Approximate  missile  flight  time  (seconds) 

E.  Approximate  probability  of  hit 

F.  What  I'actonfs)  led  to  the  selection  of  the  mode  of  attack 
indicated  in  pari  (A)  above? 

TS 1 

Ik  * _ . _ 
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Summary  Text  of  Session  Plan  Item  A.  5.  b 
Instructions  for  Preparing  Route  Sketches 

I am  now  handing  each  of  you  the  materials  that  you  will  need  in  prepar- 
ing your  first  route  sketch.  As  you  can  see  there  is  a form  called  the  Route 
Description  Form  and  there  is  a piece  of  clear  acetate.  The  route  description 
form  defines  the  route  you  are  to  select  while  the  acetate  is  to  be  used  to  pre- 
pare your  route  sketch. 

First,  I want  each  of  you  to  attach  the  clear  acetate  to  your  map  board 
using  the  tape  I am  passing  around  now.  You  will  notice  that  the  acetate  has 
markings  to  indicate  where  the  corners  of  the  battlefield  area  are  to  be  placed 
under  the  overlay.  You  will  also  notice  that  the  overlay  is  marked  with  your 
subject  number  and  route  number,  in  this  case  route  number  one. 

Now  I want  you  to  examine  the  route  description  form.  First,  you  will 
see  that  I have  filled  in  all  the  blanks  above  the  dotted  line.  The  subject  num- 
ber and  route  number  have  been  recorded,  and  other  information  is  provided 
to  define  the  route  selection  situation.  First,  the  starting  coordinates  of  the 
route  are  provided  and  you  should  make  sure  that  your  route  actually  does 
start  from  the  specified  point.  For  this  first  route  each  of  your  starting  points 
will  be  somewhere  on  the  western  boundary  of  the  battlefield. 

Next,  the  terminal  position  of  the  route  is  specified.  This  position  is 
the  point  at  which  your  route  should  terminate  after  starting  from  the  specified 
initial  position  and  passing  through  the  attack  position  that  you  choose.  For 
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this  route  the  terminal  position  is  anywhere  on  the  western  boundary.  Please 


I 


I 

I 
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make  sure  that  your  route  actually  does  terminate  there. 

Next,  the  weapons  that  you  will  face  while  executing  the  route  are  speci- 
fied. Weapon  number  one  corresponds  to  the  position  labeled  "1"  on  your  map 
board,  and  the  coordinates  that  are  recorded  should  be  those  of  that  position. 
Similarly,  weapon  number  two  corresponds  to  the  position  labeled  "2".  The 
types  of  weapons  specified  have  been  selected  from  those  that  we  studied 
earlier. 

Finally,  the  weapon  position  you  are  to  attack  is  identified.  It  may  be 
either  position  number  one  or  position  number  two,  but  in  any  case  the  weapon 
that  is  not  to  be  attacked  can  be  thought  of  as  a supportin'  weapon  which  you 
must  be  aware  of. 

Now,  below  the  dotted  line  are  several  data  items  which  I want  you  to 
record  after  you  have  completed  your  route  sketch.  You  will  note  that  you  may 
select  either  hover  fire  or  running  fire,  but  I would  like  for  you  to  tell  me  what 
factors  you  considered  in  making  the  choice.  The  other  data  can  be  determined 
by  using  the  scales  we  discussed  earlier  or  by  using  the  TOW  weapon  system 
operational  data  with  which  you  have  been  provided.  Please  try  to  determine 
these  data  as  accurately  as  possible. 

In  preparing  your  route  sketch  you  may  use  the  orange  grease  pencil 
that  is  in  your  supply.  Be  sure  and  place  arrow  heads  on  the  sketch  so  that  I 
will  be  able  to  determine  your  intended  direction  of  motion  at  all  points.  Also, 
please  indicate  with  a circle  the  position  you  select  as  the  firing  point  for  your 
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attack.  If  you  need  to  make  corrections  to  your  sketch,  you  will  find  a cotton 
pad  in  your  supply  box  for  erasures. 

You  may  start  your  first  sketch  now.  For  the  remainder  of  the  session 
you  will  be  working  at  your  own  speed.  Therefore,  when  you  complete  your 
first  sketch  let  me  know.  I will  collect  your  data  and  give  you  the  material 
you  need  for  your  second  sketch.  Please  take  as  long  as  you  like  on  each 
sketch.  Also,  if  you  have  a question,  please  let  me  know. 
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Exhibit  D.  13 

Summary  Text  of  Session  Plan  Item  B.  1 
Introduction  to  Route  Evaluation  Session 

Good  morning,  gentlemen.  It  is  really  a pleasure  to  see  your  smiling 
faces  once  again.  I hope  the  early  hour  doesn't  reduce  your  enthusiasm  for  the 
task  that  is  facing  us  this  morning. 

As  you  will  recall,  about  a month  ago  we  met,  and  at  that  time  we  went 
through  a two- hour  session  in  which  we  gathered  very  specific  and  detailed  in- 
formation concerning  your  own  personal  leelings  about  helicopter  route  selec- 
tion. Since  that  time  I have  been  very  busy  reducing  that  data  and  using  it  in  a 
model  that  has  been  developed  to  predict  what  we  think  you  will  select  as  a 
route,  given  a particular  tactical  situation.  The  purpose  of  the  session  this 
morning  is  to  see  how  well  we  have  done. 

The  sketches  of  the  four  routes  you  prepared  in  the  earlier  session  are 
going  to  be  returned  to  you  shortly,  along  with  a series  of  alternative  routes 
prepared  by  the  computer  model.  There  will  be  a different  set  of  alternates 
for  each  of  the  four  route  selection  situations.  Once  you  have  this  material  in 
hand,  you  will  be  instructed  in  a method  of  evaluating  the  alternatives.  You 
will  then  analyze  the  alternatives  according  to  the  instructions,  and  finally 
you  will  record  your  evaluations  on  forms  that  I will  provide. 

At  this  point  let  me  reiterate  a statement  I made  several  times  during 
our  first  session  which  is  just  as  appropriate  today.  We  are  not  attempting 
to  judge  your  skill  as  an  aviator  in  this  experiment.  Therefore,  there  are  no 
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right  or  wrong  answers.  Instead,  we  want  your  opinions  as  highly  qualified 
military  aviators.  We  want  your  honest  evaluation  of  the  alternatives  relative 
to  the  routes  you  selected.  With  this  information  we  will  have  a yardstick  by 
which  our  model  may  be  judged.  We  will  then  be  able  to  decide  either  that  we 
have  an  adequate  model  or  that  we  must  continue  to  refine  the  model.  Your 
conscientious  cooperation  in  this  exercise  is  therefore  very  critical  to  our 
success. 
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Exhibit  D.  14 

Description  oi  Materials  Used  in  Session  Plan  Item  B.  2 

The  material  packet  handed  to  each  subject  at  the  beginning  of  the  Route 
Evaluation  Session  contained  the  following  31  items: 

1.  Four  (4)  acetate  overlays; 

2.  Four  (4)  map  sheets; 

3.  Four  (4)  answer  sheets; 

4.  Sixteen  (16)  translucent  overlays; 

5.  One  (1)  TOW  Weapon  System  Description; 

6.  One  (1)  felt  tip  marker;  and 

7.  One  (1)  lead  pencil. 

Examples  of  the  translucent  overlays  and  answer  sheets  appear  at  Exhibit  D.  15 
while  the  TOW  Weapon  System  Description  appears  at  Exhibit  D- 3.  The  map 
board  at  Exhibit  D.  6 is  similar  in  appearance  to  the  map  sheets  in  the  packet. 
However,  because  of  format  requirements  all  map  sheets  and  overlays,  as 
presented,  are  reduced  in  size  from  their  original  dimensions  of  7 1/2  inches 
x 14  1/2  inches.  Furthermore,  the  original  coloring  schemes,  which  were  of 
benefit  to  the  subjects  during  the  experiment,  were  lost  during  the  reduction 
process. 

Descriptions  of  the  use  of  the  materials  and  of  the  coloring  schemes  used 
arc  presented  at  Exhibits  D.  16  and  I).  17. 
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Exhibit  D.  15 


Examples  of  Materials  in  Session  Plan  Item  B.  2 


Route  Evaluation  Form 


Subject  Number 


Route  Number 


Route 

Ranking*  * 

Alternative 

Indifferent 

Almost 

Mildly 

Strongly 

Number 

Indifferent 

Prefer 

Prefer 

(100) 

(95-99) 

(70-94) 

(0-69) 

size 


Original  size  of  map:  7.  5 x 14.  5 inches 


size 


Exhibit  D.  16 

Summary  Text  of  Session  Plan  Item  B.  3.  a 
Instructions  for  Transferring  Routes 


The  package  I have  just  handed  you  contains  a variety  of  materials  that 
will  be  used  during  the  remainder  of  this  morning's  session.  You  will  notice 
that  among  these  materials  are  the  four  acetate  route  overlays  that  you  pre- 
pared for  me  during  our  first  session.  Also,  there  are  four  map  sheets  that 
correspond  to  the  tactical  situations  that  you  faced  in  preparing  each  of  the 
four  route  overlays.  As  you  can  see,  these  map  sheets  actually  correspond  to 
the  map  boards  that  you  used  during  the  first  session. 
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The  first  item  of  business  that  we  wish  to  accomplish  this  morning  is  to 
transfer  the  routes  from  the  acetate  overlays  to  the  four  map  sheets  I have  pro- 
vided. There  are  actually  two  reasons  for  this  exercise.  The  first  reason  is 
that  we  wish  to  refamiliarize  you  with  the  routes  that  you  picked  and  also  with 
the  situations  that  you  faced.  The  second  purpose  of  the  exercise  is  to  prepare 
a map  board  which  has  your  route  registered  on  it,  and  which  may  be  used  later 
in  evaluating  the  route  alternatives  that  have  been  prepared  by  the  computer. 

As  you  transfer  each  of  your  routes  from  your  acetate  overlays,  I want 
you  to  be  very  careful  in  transfering  the  route  to  make  sure  that  the  map  sheet 
that  you  use  actually  corresponds  to  the  tactical  situation  that  is  associated  with 
the  particular  route.  Your  route  overlays  and  the  map  sheets  are  labeled  so 
that  you  should  be  able  to  do  this  with  no  difficulty.  However,  I will  monitor 
your  progress  to  make  sure  that  there  are  no  problems.  Please  let  me  know  if 
you  have  any  (fues  lions. 
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Exhibit  D.  17 


Summary  Text  of  Session  Plan  Item  B.  4.  a 

Instructions  for  Using  Route  Overlays  and  Answer 
Sheets  in  Performing  Route  Evaluations 

Now  that  we  have  the  map  sheets  with  the  routes  recorded  on  them,  I 
want  you  to  take  a look  at  the  other  materials  I have  provided  in  your  package. 
First,  you  will  find  there  are  four  translucent  plots,  or  overlays,  that  have  been 
prepared  by  the  computer  for  each  of  your  four  routes.  Furthermore,  on  each 
of  these  translucent  overlays  you  will  find  plots  of  three  alternative  routes. 
During  the  remainder  of  the  session  this  morning,  we  are  going  to  be  comparing 
the  alternative  routes  that  appear  on  these  translucent  overlays  with  the  routes 
that  you  have  just  t-aced  on  the  situation  map  sheets.  Each  of  the  overlays  is 
labeled  so  that  you  will  be  able  to  identify  it  with  a particular  route  that  you 
chose  during  our  first  session. 

Now,  let  us  take  our  first  route  map  sheet  and  the  four  computer  over- 
lays associated  with  it.  You  will  notice  if  you  lay  the  first  overlay  over  your 
map  sheet  you  will  be  able  to  do  a comparison  between  the  route  you  picked  and 
the  three  alternative  routes  that  appear  on  the  overlay.  The  procedure  will  be 
to  prepare  some  information  about  your  feelings  with  regard  to  the  three  alter- 
natives that  appear  on  this  first  overlay.  You  will  then  select  the  second  over- 
lay for  route  1 followed  by  the  third  overlay  for  route  1.  Then  you  will  progress 
to  your  route  2 and  repeat  the  process.  The  procedure  will  continue  until  all 
lour  of  your  routes  have  been  analyzed. 
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Now,  I have  also  included  four  answer  sheets  in  the  package;  one  answer 
sheet  for  each  route  and  its  alternatives.  On  this  answer  sheet  you  will  record 
the  data  that  I am  going  to  explain.  What  we  want  you  to  do  is  to  compare  each 
alternative  to  the  route  which  you  selected,  giving  us  your  own  subjective  evalu- 
ation of  the  alternative  with  respect  to  the  route  you  picked.  For  this  purpose 
the  answer  sheet  indicates  that  we  are  going  to  allow  you  to  classify  an  alter- 
native in  any  one  of  four  categories. 

The  first  category  is  the  one  in  which  you  feel  that  there  is  absolutely 
no  difference  between  the  alternative  and  the  route  you  picked.  In  this  situation 
you  would  have  to,  for  example,  toss  a coin  to  decide  whether  you  would  use  the 
route  you  picked  or  the  particular  alternative. 

The  second  category  we  have  labeled  as  being  almost  indifferent. 

Actually,  such  a label  is  bad  terminology.  What  we  really  mean  is  that  should 
an  alternative  fall  in  this  category  you  would  be  unable  to  cite  any  particular 
reason  for  the  alternative  being  preferred  to  your  route  or  vice  versa,  but  at 
the  same  time  you  would  still  feel  that  one  route  is  preferred  over  the  other. 

The  last  two  categories  are  included  for  those  situations  in  which  you 
can  definitely  cite  some  reason  for  a preference  to  exist,  either  a preference 
for  your  route  or  a preference  for  the  alternative  route.  Whether  an  alternative 
falls  into  the  third  category  or  the  fourth  category  is  strictly  a matter  of  degree. 
You  would  classify  those  situations  in  which  you  mildly  prefer  one  route  to  an- 
other in  the  third  category.  In  the  fourth  category  you  would  record  those  situ- 
ations in  which  you  strongly  prefer  one  route  to  the  other.  We  presume  that  a 
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route  comparison  would  fall  in  the  fourth  category,  if,  for  example,  the  computer 
alternative  flagrantly  violated  some  particular  criterion  that  you  use  when  you 
pick  a route. 

Once  a particular  route  comparison  has  been  made  between  your  route 
and  one  of  the  alternatives,  and  once  you  have  entered  its  qualitative  evaluation 
on  the  answer  sheet,  we  want  you  to  actually  rate  the  "non-preferred"  route  on 
a scale  between  zero  and  one  hundred  to  indicate  to  us  the  degree  of  feeling  that 
you  have  about  the  comparison  that  you  have  made.  This  scale  has  been  divided 
into  four  regions  for  you  to  use  in  your  evaluation,  with  each  region  being 
associated  with  a particular  qualitative  evaluation. 

The  first  region  is  the  one  that  includes  routes  that  are  definitely  the 
same  to  you.  A route  alternative  which  you  feel  is  definitely  the  same  would  be 
rated  at  one  hundred.  Next,  for  situations  in  which  you  are  almost  indifferent, 
you  will  select  a value  between  ninety- five  and  ninety-nine  to  indicate  your  de- 
gree of  preference.  The  number  you  select  is  the  rating  of  the  non- preferred 
route.  For  example,  were  you  to  feel  that  a route  is  almost  as  good  as  your 
own  route,  you  might  rate  the  computer  route  at  ninety- seven.  Consequently, 
you  would  use  the  second  column  of  the  answer  sheet  to  indicate  that  you  are 
almost  indifferent,  entering  an  "M"  to  indicate  that  you  actually  prefer  your 
own  route.  Then,  in  column  five,  you  would  enter  the  rating  that  applies  to  the 
non-preferred  route,  in  this  case  ninety- seven. 

Ff  you  mildly  prefer  one  route  to  another, you  rate  the  non-preferred 
route  in  column  live  with  a number  between  seventy  and  ninety- four.  And,  in 
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the  third  column  of  the  answer  sheet,  you  enter  "M"  if  you  prefer  your  own 
route  or  "C"  if  you  prefer  the  computer  route.  Similarly,  if  you  strongly  prefer 
one  route  to  another  you  rate  the  non-preferred  route  in  column  five  with  a num- 
ber between  zero  and  sixty-nine.  Then  you  enter  an  "M"  or  a "C"  in  column 
four  to  indicate  which  route  is  preferred. 

Now,  in  examining  the  translucent  route  overlays,  you  will  notice  that 
each  alternative  is  labeled  with  a number.  This  same  number  appears  on  the 
answer  sheets,  and  you  will  note  that  there  can  be  as  many  as  twelve  alterna- 
tives. However,  you  may  not  find  twelve  alternatives  for  each  route.  The 
reason  for  this  is  that  the  computer  sometimes  duplicates  routes.  If  such 
duplication  exists,  you  will  find  that  the  same  alternative  number  appears  for 
each  of  the  duplicate  routes.  So,  all  you  have  to  do  is  perform  the  number  of 
comparisons  indicated  on  your  answer  sheets.  In  general,  this  number  will  be 
between  eight  and  ten. 

Next,  you  will  notice  that  each  alternative  is  labeled  with  sufficient 
arrow  heads  to  indicate  direction  of  motion  at  all  points  a long  the  route.  You 
will  also  notice  that  the  firing  point  in  each  route  is  labeled  with  a red  circle. 
Finally,  you  will  notice  that  half  of  the  alternatives  are  hover  fire  alternatives 
and  half  of  the  alternatives  are  running  fire  alternatives.  We  prepared  both 
types  of  alternatives  without  regard  to  type  of  fire  you  selected  during  our 
original  session.  We  want  you  to  rate  all  of  these  alternatives,  taking  into 
consideration  the  factors  you  used  in  selecting  one  lyin'  of  fire  or  another. 
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As  you  enter  your  answers  on  the  answer  sheet  you  are  providing  us 
both  with  quantitative  and  with  qualitative  information.  However,  there  is 
additional  qualitative  information  that  you  can  provide  which  may  be  extremely 
valuable  to  us.  If  you  feel  so  disposed,  we  would  like  you  to  actually  tell  us 
with  comments,  why  you  prefer  one  route  over  another.  This  information  may 
be  especially  valuable  for  those  situations  in  which  you  mildly  prefer  or  strong- 
ly prefer  your  original  route  to  one  of  the  computer  alternatives  or  vice  versa. 
These  qualitative  statements  may  be  of  use  in  isolating  particular  facets  of  the 
route  selection  problem  in  which  the  computer  model  does  not  do  a good  job. 

We  could  then  use  the  information  to  revise  the  route  selection  model  to  do  a 
better  job. 

Remember,  there  are  no  right  or  wrong  answers  to  the  questions  we 
are  asking.  We  want  you  to  give  us  your  own  personal  reactions.  You  will 
never  be  judged  in  this  experiment  on  the  basis  of  your  piloting  capabilities. 

We  strictly  want  information  that  reveals  to  us  how  helicopter  pilots  think. 

You  may  start  with  your  evaluations  now  and  you  may  continue  until  you 
are  finished.  If  there  are  any  questions,  I will  be  available  to  answer  them. 
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1 APPENDIX  E 

PHASE  H EXPERIMENTAL  DATA 
AND  ANALYSES 
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In  this  appendix  the  following  abbreviations  apply: 

Threat  Perception  Model 

AT  - Acid  Test  Model 
1 - Model  One 

3 - Model  Three 

Attack  Mode 

HF  - Hover  Fire 
RF  - Running  Fire 

Data  Set 

1 - Results  produced  by  using  personalistic  data  from  each 

subject 

2 - Results  produced  by  using  averaged  or  "super  subject" 

data  for  each  subject 
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Table  E.  1 


Alternative  Numbering  Scheme 


Personal  Data 

Super  Subject  Data 

AT 

1 

3 

AT 

1 

3 

Subiect 

Route 

RF 

HF 

RF 

HF 

RF 

HF 

RF 

HF 

RF 

HF 

RF 

HF 

6 

1 

1 

4 

2 

5 

4 

1 

4 

2 

5 

3 

4 

2 

7 

8 

3 

6 

5 

1 

4 

3 

6 

2 

5 

3 

7 

9 

3 

6 

8 

10 

1 

4 

3 

6 

2 

5 

4 

7 

9 

3 

6 

8 

10 

1 

4 

3 

6 

2 

5 

7 

1 

7 

4 

9 

5 

8 

6 

1 

4 

3 

5 

2 

6 

2 

7 

9 

3 

6 

8 

10 

1 

4 

3 

6 

2 

5 

3 

7 

10 

9 

6 

8 

11 

1 

4 

3 

6 

2 

5 

4 

7 

9 

3 

6 

8 

10 

1 

4 

3 

6 

2 

5 

8 

1 

1 

3 

2 

5 

1 

3 

1 

3 

2 

4 

1 

3 

2 

2 

8 

3 

9 

7 

5 

1 

4 

3 

G 

2 

5 

3 

7 

8 

3 

6 

7 

9 

1 

4 

3 

6 

2 

5 

4 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

9 

1 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

2 

7 

10 

9 

6 

8 

11 

1 

4 

3 

6 

2 

5 

3 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

4 

7 

8 

3 

6 

7 

9 

1 

4 

3 

6 

2 

5 

10 

1 

6 

8 

2 

5 

7 

4 

1 

4 

2 

5 

3 

4 

2 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

3 

7 

9 

3 

6 

8 

1 

4 

3 

G 

2 

5 

4 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

11 

fff 

1 

4 

3 

5 

7 

6 

1 

4 

3 

5 

2 

6 

u 

1 

5 

3 

6 

7 

5 

1 

4 

3 

6 

2 

5 

IS 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

n 

7 

9 

3 

G 

8 

1 

4 

3 

6 

2 

5 

12 

E9 

1 

3 

2 

5 

1 

3 

1 

3 

2 

4 

1 

3 

2 

7 

5 

3 

8 

7 

5 

1 

4 

3 

6 

2 

5 

3 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

4 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

13 

1 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

2 

7 

9 

3 

6 

8 

1 

4 

3 

6 

2 

5 

3 

7 

9 

3 

6 

8 

1 

4 

3 

G 

2 

5 

4 

7 

9 

3 

6 

8 

1 

4 

3 
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Table  E.  2 


Summary  of  Similarity  in  Routes  Produced  by  Two  Data  Sets 


AT 

i 1 

^ 3 

% 

Totals 

Subject 

Route 

RF 

HF 

RF 

HF 

RF 

HF 

Similarity 

<%) 

6 

1 

1 

1 

n 

■1 

1 

1 

100 

2 

0 

0 

1’ 

1 

1 

67 

3 

0 

0 

u 

18818 

0 

0 

33 

4 

0 

0 

li 

1 

0 

0 

33 

58 

7 

1 

0 

1 

0 

0 

1 

50 

2 

0 

0 

■1 

■ 

0 

0 

33 

3 

0 

0 

0 

1 1 

0 

0 

17 

4 

0 

0 

MM 

0 

0 

33 

33 

8 

1 

1 

1 

1 

0 

1 

1 

83 

2 

0 

0 

1 

0 

0 

1 

33 

3 

0 

0 

1 

m 

0 

0 

33 

4 

0 

0 

1 

El 

0 

0 

33 

46 

11 

1 

1 

1 

1 

BE 

0 

1 

83 

2 

1 

0 

1 

0 

1 

67 

3 

0 

0 

1 

if 

0 

0 

33 

4 

0 

0 

1 

■I 

0 

0 

33 

54 

12 

1 

1 

1 

1 

0 

1 

1 

83 

2 

0 

0 

1 

0 

0 

1 

33 

3 

0 

0 

1 

H 

0 

0 

33 

4 

0 

0 

1 

n 

0 

0 

33 

46 

13 

1 

0 

0 

1 

n 

0 

0 

33 

2 

0 

0 

1 

11 

0 

0 

33 

3 

0 

0 

1 

H 

0 

0 

33 

4 

0 

0 

1 

KB 

0 

0 

33 

33 

Totals  (%) 

21 

21 

92 



83 

LiZ_j 

38 

45 

1 - route  produced  by  data  set  1 is  the  same  as  that  produced  by  data  set  2 
0 - different  routes  are  produced 
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Table  E.  3 


Summary  of  Similarity  in  Routes  Produced  by  Acid 
Test  Model  and  Model  Three 


% Totals 

RF  | HF  [Similarity  (%) 


Totals  (%) 


Data 

Set  1 

% 

Totals 

RF 

HF 

Similarity 

(%) 

0 

1 

50 

0 

0 

0 

0 

0 

0 

0 

0 

0 

12 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

100 

0 

0 

0 

1 

0 

50 

0 

0 

0 

38 

0 

0 

0 

0 

1 

50 

0 

0 

0 

0 

0 

0 

12 

II 

11 

100 

100 

0 

0 

o 

0 

0 

0 

50 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

17 

21 

19 

1 - route  produced  by  Acid  Test  Model  is  the  same  as  that  produced 
by  Model  Three 


0 - different  routes  are  produced 


Table  E.  4 


Summary  of  Similarity  in  Route  Alternatives  Presented 
to  Comparable  Subjects  (Data  Set  1) 


Subject 

Pair 

Route 

First  Subject 
Alternatives 

Second  Subject 
Alternatives 

% 

Similarity 

1 

2 

3 

4 

5 

6 

1 

2 

3 

4 

5 

6 

(7, 11) 

1 

1 

1 

1 

1 

1 

1 

1 

1 

67 

2 

1 

1 

1 

1 

1 

1 

50 

3 

1 

1 

1 

1 

33 

4 

0 

1 

1 

1 

1 

33 

(8, 12) 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

100 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

83 

3 

1 

1 

0 

0 

1 

1 

33 

4 

1 

1 

0 

0 

1 

1 

33 

Total  for  (7, 11)  (% 


46 


Total  for  (8, 12)  (%) 
Overall 


62 


1 - at  least  one  route  alternative  of  paired  subject  is  the  same  as  the 
alternative  in  question 


0 - no  comparable  route  alternative  exists  for  paired  subject 
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Table  E.  5 

Alternative  Route  Ratings 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1-Data  Set 

1 

243.  00 

243.  00 

O2  + 24  O2 

2- Route 

3 

60090.92 

20030. 30 

2 2 

0 + 12  0n 
e 2 

*** 

3-  Perception 

2 

3552.  88 

1776.  44 

al  + 16  °3 

*** 

Model 

4- Attack  Mode 

1 

1704.08 

1704.08 

a2  + 24  of 
e 4 

** 

12 

3 

1107.00 

369.00 

„2  , c „2 
°e+  6 °12 

13 

2 

1478.  62 

739. 31 

02  + 8 02 

* 

e 13 

14 

1 

408.  33 

408.  33 

C2  + 12  02 

e 14 

23 

6 

8220.96 

1370. 16 

i* 4 i3 

*** 

24 

3 

1777. 59 

592.  53 

„2  , a „2 
°e+  6°24 

* 

34 

2 

9.  54 

4.  77 

o2  + 8 a2 

e 34 

Residual 

23 

7905.  92 

256.  78 

o2 

e 

Total 

47 

86499.00 

* Significant  at  a - 0. 10 
* * Significant  at  a = 0.  05 
***  Significant  at  a = 0.01 


Figure  E.  1. — Analysis  of  Variance  for  Subject  6 
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Source  of 

Degrees  of 

Sums  of 

Variation 

Freedom 

Squares 

1-Data  Set 

1 

2508.52 

2- Route 

3 

8229.  73 

3- Perception 
Model 

2 

4876.04 

4- Attack  Mode 

1 

1.  69 

12 

3 

1614.  73 

13 

2 

3563. 30 

14 

1 

1.02 

23 

6 

5486.  96 

24 

3 

126.90 

34 

2 

315. 88 

Residual 

23 

5845.  66 

Total 

47 

32570.  40 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

2508.52 

o2  + 24  02 
e 1 

*** 

2743.  24 

a2  + 12  o2 
e 2 

*** 

2438.02 

d2  + 16  o2 
e 3 

*** 

1.  69 

a2  + 24  c2 
e 4 

538. 24 

a2  + 6 a2 
e 12 

1781.  65 

a2  + 8 o2 
e 13 

*** 

1.02 

°e  + 12 

914.  49 

°e+  4 ’23 

** 

42.  30 

n2  + 6 n2 
e 24 

157. 94 

n2  + 8 o2 
e 34 

254.  16 

n2 

e 

* Significant  at  a = 0. 10 
**  Significant  at  a - 0.05 
***  Significant  at  a = 0. 01 


Figure  E.  2. — Analysis  of  Variance  for  Subject  7 
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Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1-Data  Set 

1 

396.75 

396.  75 

o2  + 24  a2 
e 1 

** 

2- Route 

3 

12801.  14 

4267. 04 

a2  + 12  a2 
e 2 

*** 

3- Perception 
Model 

2 

9193.02 

4596.51 

°e  + 16  °3 

*** 

4- Attack  Mode 

1 

396.  75 

396.  75 

O2  + 24  C2 
e 4 

** 

12 

3 

663. 11 

221.  04 

o2  + 6 O2 
e 12 

** 

13 

2 

682.  65 

341. 32 

O2  + 8 O2 

e 13 

*** 

14 

1 

56.33 

56.  33 

a2  + 12  a2 
e 14 

23 

6 

6691.51 

1115. 25 

02  + 4 C2 
e 23 

*** 

24 

3 

300. 78 

100. 26 

_2  , c „2 
°e+  6 °24 

34 

2 

223.  65 

111. 83 

02  + 8 O2 

e 34 

Residual 

Total 

23 

47 

1339.94 
32745.  61 

58.  26 

o2 

e 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0.  01 


Figure  E.3. — Analysis  of  Variance  for  Subject  8 
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Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1-Data  Set 

1 

75.  00 

75.00 

o2 

e 

+ 24  o2 

2-  Route 

3 

7925. 42 

2641. 81 

c2 

e 

+ 12  4 

*** 

3- Perception 
Model 

2 

9522. 16 

4761.07 

02 

e 

+ 16  o2 
3 

*** 

4- Attack  Mode 

1 

52.  08 

52.08 

o2 

e 

+ 24  a2 
4 

12 

3 

1076.  50 

358. 83 

o2 

e 

+ 6 a2 
12 

13 

2 

98.01 

49.01 

e 

+ 8 a2 
13 

14 

1 

176.  33 

176.  33 

02 

e 

+ 12  02 
14 

23 

6 

14946.  35 

2491.06 

o2 

e 

+ 4 02 
23 

*** 

24 

3 

915.  42 

305. 14 

o2 

e 

+ 6 C2 
24 

34 

2 

2650.  18 

1325. 09 

c2 

e 

+ 8 02 
34 

* 

Residual 

23 

11348. 40 

493. 41 

d2 

e 


Total  47  48785.  83 


* Significant  at  a = 0. 10 
**  Significant  at  a - 0.  05 
***  Significant  at  a - 0.01 


Figure  E.  4. --Analysis  of  Variance  for  Subject  11 
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Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1-Data  Set 

1 

540.02 

540.02 

G2  + 24  o2 
e 1 

2-  Route 

3 

6246.90 

2082.  30 

C2  + 12  c2 
e 2 

*** 

3- Perception 

2 

23325.08 

11662. 54 

a2 + 16  o§ 

*** 

Model 

4- Attack  Mode 

1 

3553.51 

3553.51 

G2  + 24  o2 
e 4 

*** 

12 

3 

2231.  40 

743.  80 

„2  , c _2 
0 + 60 

e 12 

13 

2 

3140.  74 

1570. 37 

O2  + 8 O2 
e 13 

** 

14 

1 

553.53 

553. 53 

d2  + 12  o2 
e 14 

23 

6 

9681.  25 

1613.54 

o2  + 4 a2 
e 23 

*** 

24 

3 

2486. 25 

828.75 

,2  , c „2 
oe  + 6 o24 

* 

34 

2 

240. 74 

120. 37 

02  + 8 O2 

e 34 

Residual 

23 

7698.01 

334.  70 

G2 

e 

Total 

47 

59697.40 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.05 
***  Significant  at  a = 0.  01 


Figure  E.  5. — Analysis  of  Variance  for  Subject  12 
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Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1-Data  Set 

1 

2821.  32 

2821. 32 

o2  + 24  02 
e 1 

* 

2- Route 

3 

10505.74 

3501.91 

a2  + 12  a2 
e 2 

** 

3-  Perception 
Model 

2 

23004. 05 

11502.  03 

O2  + 16  C2 
e 3 

*** 

4- Attack  Mode 

1 

6.  75 

6.75 

a2  + 24  c2 
e 4 

12 

3 

136.  20 

45.  40 

*2  -i.  a ri2 

°e  b 12 

13 

2 

2815. 25 

1407.  62 

a2  + 8 a2 
e 13 

14 

1 

3468.  01 

3468.  01 

i * 12 

* 

23 

6 

5628.  62 

938. 10 

°e+  4023 

24 

3 

5852.  43 

1950. 81 

.2  , ..  „2 
0 + 6 0„, 
e 24 

34 

2 

978.  60 

489.  30 

°e  + SOt< 

Residual 

23 

20074.  72 

872.  81 

0* 

e 

Total 

47 

75291.  62 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.05 
***  Significant  at  a = 0.  01 


Figure  E.6. — Analysis  of  Variance  for  Subject  13 


I 1 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1- Route 

3 

36510.46 

12170. 15 

„2  ^ „ 2 
ae+  6 o1 

*** 

2- Perception 
Model 

2 

1396.  75 

698.  38 

O2  + 8 O2 

e 2 

3- Attack  Mode 

1 

1890. 38 

1890. 38 

°e  + 12  °i 

** 

12 

6 

6561.92 

1093. 65 

o 1+  2 '>12 

13 

3 

2074.  46 

691.  49 

°e+  2 °13 

23 

2 

982.75 

491.38 

02  + 4 O2 
e 23 

Residual 

6 

1525.92 

254.  32 

o2 

e 

Total 

23 

50942. 62 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0.01 


Figure  E.  7 . — Analysis  of  Variance  for  Subject  6 

Data  Set  1 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1 - Route 

3 

3723. 46 

1241. 15 

a2  + 6 a2 
e 1 

*** 

2 -Perception 
Model 

2 

441. 75 

220. 88 

a2  + 8 a2 
e 2 

3- Attack  Mode 

1 

0.04 

0.04 

a2  + 12  a2 
e 3 

12 

6 

2856.92 

476. 15 

o2  + 2d2 
e 12 

** 

13 

3 

178.46 

59.  45) 

i - 3 

23 

2 

267.58 

133. 79 

°e+  4 is 

Residual 

6 

514.  42 

85.74 

o2 

e 


Total  23  7982.62 


* Significant  at  a = 0. 10 
**  Significant  at  a = 0.05 
***  Significant  at  a - 0. 01 


Figure  E.  8 . — Analysis  of  Variance  for  Subject  7 

Data  Set  1 


Source  of 

Degrees  of 

Sums  of 

Variation 

Freedom 

Squares 

1- Route 

3 

8444.  79 

2 -Perception 
Model 

2 

3142.58 

3- Attack  Mode 

1 

77.04 

12 

6 

3955.09 

13 

3 

267. 80 

23 

2 

39.58 

Residual 

6 

86.04 

Total 

23 

16012.90 

Mean 

Expected 

Squares 

Mean  Squares 

Significance 

2814.  93 

a2  + 6 c? 
e i 

*** 

1571. 29 

a2  + 8 a2 
e 2 

*** 

77.04 

a2  + 12  a2 
e 3 

* 

659. 18 

°e+  2°12 

*** 

89.  27 

°e+  3°U 

** 

19.  79 

a2  + 4 o2 
e 23 

14.  34 

d2 

e 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.01 


Figure  E.  9 . — Analysis  of  Variance  for  Subject  8 

Data  Set  1 


<11  <1 


I 

I 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squan 

1 - Route 

3 

3898.  79 

1299.  60 

o2  + 6 cr2 
e 1 

2 -Perception 
Model 

2 

4064.  08 

2032.  04 

O2  + 8 O2 

e 2 

3- Attack  Mode 

1 

18.38 

18.38 

a2  + 12  c2 
e 3 

12 

6 

5632. 59 

938.76 

o2  + 2 o2 
e 12 

13 

3 

162.79 

54.  26 

a2  + 3 a2 
e 13 

23 

2 

698.  25 

349.  12 

at*  4i3 

Residual 

6 

6575.04 

1095. 84 

e 

Total 

23 

21049.91 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0. 01 


Significance 


Figure  E.  10. — Analysis  of  Variance  for  Subject  11 

Data  Set  1 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1 - Route 

3 

804.  83 

268. 28 

o2 

e 

+ 

6 O2 
1 

2 -Perception 

2 

21760.07 

10880.03 

2 

0 

+ 

8 a2 

*** 

Model 

e 

2 

3- Attack  Mode 

1 

3456.  00 

3456.  00 

o2 

e 

+ 12  0 

3 

*** 

12 

6 

3464.  93 

577.49 

o2 

+ 

2 O2 

* 

e 

12 

13 

3 

2754.  33 

918.  11 

o2 

e 

+ 

3 O2 
13 

** 

23 

2 

720.77 

360. 38 

c2 

e 

+ 

4 C2 
23 

Residual 

6 

10y8. 86 

183. 14 

o2 

e 

Total 

23 

34059. 79 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0. 01 


Figure  E.  11. — Analysis  of  Variance  for  Subject  12 

Data  Set  l 


41(i 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Square 

1- Route 

3 

5154.  46 

1718. 15 

o2  + 
e 

6 C2 

2-Perception 

Model 

2 

13497.  58 

6748. 79 

o2  + 
e 

«°2 

3- Attack  Mode 

1 

1890.37 

1890.  37 

a2  + 12  <r 

e 3 

12 

6 

2354.  42 

392.  40 

c2  + 
e 

2 O2 
2 °12 

13 

3 

3661. 13 

1220. 38 

c2  + 
e 

3 02 
2 °13 

23 

2 

3451.76 

1725. 88 

c2  + 
e 

4 C2 
23 

Residual 

6 

10230. 20 

1705.03 

o2 

e 

Total 

23 

40239.91 

* Significant  at  a = 

0.10 

**  Significant  at  a = 

0.05 

***  Significant  at  a - 

0.01 

Figure  E.  12. — Analysis  of  Variance  for  Subject  13 

Data  Set  1 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1 - Route 

3 

24687. 46 

8229. 15 

„2  , a „2 

0 + 60 

e 1 

*** 

2 -Perception 
Model 

2 

3634.  75 

1817.38 

°e+  802 

** 

3- Attack  Mode 

1 

222.04 

222.04 

C2  + 12  C2 
e 3 

12 

6 

3671.92 

611.99 

o2  + 2 o2 
e 12 

13 

3 

224. 13 

74.71 

O2  + 3 O2 
e 13 

23 

2 

1131.08 

565.54 

02  + 4 02 
e 23 

Residual 

6 

1742.  25 

290. 38 

o2 

e 

Total 

23 

35313. 62 

I 

I 

I 


* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0.01 


Figure  E.  13. — Analysis  of  Variance  for  Subject  6 

Data  Set  2 


I 

I 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1 - Route 

3 

6121.00 

204''.  33 

a2  + 6 o2 
e 1 

* 

2 -Perception 
Model 

2 

7997. 58 

3998. 79 

a2  + 8 a2 
e 2 

** 

3- Attack  Mode 

1 

2.  67 

2.  67 

o2  + 12  o2 
e 3 

12 

6 

4145.75 

690. 96 

a2  + 2 a2 
e 12 

13 

3 

39.01 

13.00 

a2  + 3 c2 
e 13 

23 

2 

259. 09 

129. 54 

°e  + 4 °23 

Residual 

6 

3514. 21 

585.70 

c2 

e 

Total 

23 

22079.  29 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.01 


Figure  E.  14. — Analysis  of  Variance  for  Subject  7 

Data  Set  2 


419 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1- Route 

3 

5019. 46 

1673. 15 

J2  , „ _2 

0 + 60. 
e 1 

*** 

2 -Perception 
Model 

2 

6733.08 

3366.54 

02  + 8 02 
e 2 

*** 

3- Attack  Mode 

1 

376.04 

376.  04 

o2  + 12  O2 
e 3 

** 

12 

6 

3543.92 

590. 65 

02  + 2 02 
e 12 

*** 

13 

3 

117. 79 

39.  26 

°e+  3°?3 

23 

2 

223.  09 

111.  54 

°e  + 4 4) 

Residual 

6 

322.  54 

53.  76 

0 

Total  23  16335.91 


* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0. 01 


Figure  E.  15. — Analysis  of  Variance  for  Subject  8 

Data  Set  2 


420 


r 


I 

I 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1 -Route 

3 

5103. 12 

1701.04 

o2  + 

6 0. 

*** 

e 

1 

2 - Perception 

2 

5556.08 

2778.04 

a2  + 

8 a2 

** 

Model 

e 

2 

3- Attack  Mode 

1 

210. 04 

210.  04 

cr  + 12 

e 3 

12 

6 

10860. 25 

1810.  04 

a2  + 

2 

** 

e 

12 

13 

3 

1475.  80 

491.93 

°2e  + 

3 °13 

23 

2 

2460.  59 

1230.  29 

o2  + 

4 02 

* 

e 

23 

Residual 

6 

1995.05 

332.  51 

c2 

e 

Total 

23 

27660.93 

* Significant  at  a = 0. 10 
**  Significant  at  a - 0.  05 
***  Significant  at  a = 0.01 


Figure  E.  16. — Analysis  of  Variance  for  Subject  11 

Data  Set  2 


I 


I 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1 - Route 

3 

7673.  46 

2557.  82 

02 

e 

+ 6 a2 

** 

2 -Perception 
Model 

2 

4705.75 

2352.  88 

c2 

e 

+ 

** 

3- Attack  Mode 

1 

651.04 

651.04 

a2 

e 

+12»! 

12 

6 

7003.92 

1167.32 

a2 

e 

+ 2 a2 
12 

* 

13 

3 

3303. 13 

1101.04 

o2 

e 

4 3013 

* 

23 

2 

55.  09 

27.54 

4 

+ 4 °23 

Residual 

6 

1705. 25 

284.  21 

a2 

e 


Total  23  25097. 62 


* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0. 01 


Figure  E.  17. — Analysis  of  Variance  for  Subject  12 

Data  Set  2 


A 


I 

I 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1 - Route 

3 

5487.46 

1829. 15 

„2  , „ 2 
0 + 60, 
e 1 

2 -Perception 
Model 

2 

12321. 75 

6160. 88 

o|+  8 0| 

** 

3- Attack  Mode 

1 

1584.  38 

1584. 38 

02  + 12  02 
6 u 

12 

6 

3582.92 

597. 15 

* 2°212 

13 

3 

2373.  46 

791.  15 

°l*  3013 

23 

2 

365.  25 

182. 62 

O2  + 4 O2 
e 23 

Residual 

6 

6515.  42 

1085.90 

o2 

e 

Total 

23 

32230. 62 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.01 


Figure  E.  18. — Analysis  of  Variance  for  Subject  13 
Data  Set  2 


423 


1 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares  Significance 

1-Data  Set 

1 

1725.51 

1725.51 

°e  + 48 

+ 24  °u 

2-  Route 

3 

15419.78 

5139.93 

°e  4 24  °2 

+ 12  °is 

3- Subject 

1 

4280.01 

4280. 01 

a2  + 48  a2 
e 3 

4-  Perception 
Model 

2 

13676.  81 

6838.  41 

a2  + 32  o2 
e 4 

+ 16  o|4 

** 

5- Attack  Mode 

1 

36.  26 

36.  26 

a2  + 48  a2 

e 5 

+ 24  °35 

9 I 1 

12 

3 

473. 87 

157.96 

°I  + 120?2 

* 

13 

1 

858.01 

858.  01 

°e  4 24  °13 

14 

2 

2420.  40 

1210.  20 

O2  + 16  C2 
e 14 

15 

1 

102.  09 

102.09 

a2  + 24  a2 
e 15 

23 

3 

735. 37 

245. 12 

a2  + 12  a2 
e 23 

24 

6 

17734. 19 

2955. 70 

9 9 

Oe  + 8 4 

25 

3 

611. 62 

203. 87 

a2  + 12  c2 

e 25 

34 

2 

721. 40 

360.70 

a2  + 16  o2 
e 34 

35 

1 

17.  52 

17.  52 

O2  + 24  O2 
e 35 

45 

2 

2297.90 

1148.95 

„ 2 .1r  „2 
°e  + 16  °45 

★ 

Residual 

63 

24524.  69 

389.  28 

O2 

e 

Total 

95 

85635.  12 

♦Significant  at  a=  0. 10 
♦♦Significant  at  a=  0. 05 
♦♦♦Significant  at  a=  0.  01 


Figure  E.  19. — Analysis  of  Variance  for  Subjects  7 and  li 

[ ! 

424 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares  Significance 

1-Data  Set 

1 

931.  26 

931.  26 

48  of 

+ 24  02g 

*** 

2- Route 

3 

3174.  61 

1058. 20 

a2  + 
e 

24  O2 
2 

+ 12  a2 
23 

3- Subject 

1 

8990. 01 

8990. 01 

a2  + 
e 

48  02 
3 

*** 

4-  Perception 
Model 

2 

1812.  56 

906.  28 

o2  + 
e 

32  °A 

4 

+ 16  °324 

5- Attack  Mode 

1 

3162.51 

3162.51 

c2  + 
e 

48  a2 
5 

+ 24  O2 
35 

12 

3 

2260. 04 

753. 34 

o2  + 
e 

120?2 

** 

13 

1 

5.51 

5.  51 

cs2  + 
e 

240?3 

14 

2 

2675.  15 

1337. 57 

a2  + 

16  a2 

*** 

e 

14 

15 

1 

128.  34 

128.  34 

a2  + 
e 

24  is 

23 

3 

15873.  45 

5291.  15 

o2  + 
e 

12  is 

*** 

24 

6 

12452.  36 

2075.  39 

o2  + 
e 

8 0|4 

*** 

25 

3 

1847.  62 

615. 87 

a2  + 
e 

12  is 

* 

34 

2 

30705. 65 

15352. 82 

o2  + 
e 

16 

*** 

35 

1 

787. 77 

787. 77 

4* 

24  °35 

* 

45 

2 

0.  40 

0.  20 

02  + 
e 

16  O2 
45 

Residual 

63 

16624.96 

263. 89 

a2 

e 

Total 

95 

101431. 75 

♦Significant  at  a=  0. 10 
♦♦Significant  at  a=  0.  05 
♦♦♦Significant  at  a-  0.  01 


j Figure  F.  20. — Analysis  of  Variance  for  Subjects  8 and  12 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares  Significance 

1- Route 

3 

5668.  25 

1889.  42 

d2 

e 

+ 12  d2 

+ 

6°L 

2- Subject 

1 

4485.  33 

4485. 33 

«4 

+ 24  d| 

*** 

3-  Perception 
Model 

2 

3337. 17 

1668. 58 

a2 

e 

+ 16°f 

+ 

8,23 

4- Attack  Mode 

1 

8.33 

8.  33 

d2 

e 

+ 24  d2 
4 

+ 12  d2 
24 

12 

3 

1954.  00 

651.  33 

d2 

e 

+ 6,,?2 

13 

6 

7543. 50 

1257. 25 

d2 

e 

+ 4 °13 

14 

3 

127. 00 

42.  33 

d2 

e 

+ 60?4 

23 

2 

1168. 67 

584.  33 

d2 

e 

+ 8ia 

24 

1 

10.09 

10.  09 

°e 

+ 12  o|4 

34 

2 

837.  17 

418. 58 

o2 

e 

+ 8034 

Residual 

23 

8378.  36 

364.  28 

d2 

e 

Total 

47 

33517. 85 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.05 
***  Significant  at  a = 0.  01 


Figure  E.21.  — Analysis  of  Variance  for  Subjects  7 and  11 
Data  Set  1 


4 20 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 

Mean  Squares  Significance 

1- Route 

3 

2899.73 

966.58 

d2  + 
e 

12  d2  + 6 d2 
1 12 

2- Subject 

1 

4275.19 

4275.18 

d2  + 
e 

24  d2 
2 

**♦ 

3- Perception 
Model 

2 

4190.17 

2095.08 

d2  + 
e 

16  d2  + 8 d2 
3 23 

4- Attack  Mode 

1 

2282.52 

2282.52 

d2  + 
e 

24  d2  + 12  d24 

12 

3 

6349.90 

2116.63 

d2  + 
e 

6 d2 
D 12 

13 

6 

6676.83 

1112.81 

d2  + 
e 

4d2 

13 

14 

3 

2131.90 

710.63 

o|  + 

6014 

23 

2 

20712.46 

10356.23 

d2  + 
e 

8 023 

*** 

24 

1 

1250.51 

1250.51 

d2  + 
e 

12024 

*** 

34 

2 

225.17 

112.58 

of* 

8 034 

Residual 

23 

3353.55 

145.81 

e 

Total 

47 

54347.90 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.  01 


Figure  E.  22. — Analysis  of  Variance  for  Subjects  8 and  12 
Data  Set  l 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares  Significance 

1 - Route 

3 

10225.40 

3408.47 

d2 + 12  o2  + 

e 1 

6 d2 
12 

2- Subject 

1 

652.69 

652.69 

a2  + 24  o2 
e 2 

3-  Perception 

2 

12760.02 

6380.01 

a2  + 16  o|  + 

8 °23 

*** 

Model 

4- Attack  Mode 

1 

130.02 

130.02 

a2  + 24  d2  + 
e 4 

I2024 

12 

3 

998.73 

332.91 

O2  + 6 o2 
e 12 

13 

6 

11377.82 

1896.30 

°I*  4 °13 

*** 

14 

3 

928.06 

309.35 

°e*  «°14 

23 

2 

793.65 

396.82 

d2 + 8 d2 

e 23 

24 

1 

82.69 

82.69 

d2  + 12  d2 
e 24 

34 

2 

1515.57 

757.78 

d2+  8d24 

Residual 

23 

10928.27 

475.14 

d2 

e 

Total 

47 

50392.89 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a - 0. 01 


Figure  E.  23. — Analysis  of  Variance  for  Subjects  7 and  11 
Data  Set  2 


I 

I 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares  Significance 

1 - Route 

3 

2534.92 

844.97 

d2  + 
e 

12  d2  + 

8 °12 

2-  Subject 

1 

4720.33 

4720.33 

d2  + 
e 

24  0 2 
2 

*** 

3-  Perception 
Model 

2 

297.54 

148.77 

d2  + 
e 

16  °3  1 

8 °23 

4r-  Attack  Mode 

1 

1008.33 

1008.33 

d2  + 
e 

24  d2  + 
4 

12024 

*** 

12 

3 

10158.00 

3386.00 

d2  + 

e 

6 o2 
12 

*** 

13 

6 

6222.46 

1037.08 

d2  + 
e 

40L 

** 

14 

3 

1500.67 

500.22 

d2  + 
e 

°°14 

23 

2 

11141.30 

5570.64 

d2  + 
e 

8°23 

*** 

24 

1 

18.76 

18.76 

d2  + 

e 

12  Ch 

34 

2 

202.79 

101.40 

d2  + 
e 

8 d2 
34 

Residual 

23 

8348.77 

362.99 

d2 

e 

Total 

47 

46153.84 

* Significant  at  a = 0. 10 
**  Significant  at  a - 0.05 
***  Significant  at  a = 0.  01 


( 

Figure  E.  24. — Analysis  of  Variance  for  Subjects  8 and  12 
Data  Set  2 


Subject  6 


Subject  7 


Perception  Model 

Perception  Model 

/\  100 1 

- 

/ N.  80  ‘ 

• 

w 60  - 

bo 

H 

* 

M 

a 40 . 

20  - 

Subject  8 

Subject  11 

Perception  Model 


Perception  Model 


Figure  K.  27. — Plot  of  Perception  Model  Effect  by  Subject 
(Combined  Data  Sets) 
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bo 

C 
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100 
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Figure  E.  28.  — Plot  of  Attack  Mode  Effect  by  Subject 
(Combined  Data  Sets) 
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Figure  E.  30. — Plot  of  Perception  Model  Effect  by  Data  Set 

and  Subject 


35 


Attack  Mode 


Attack  Mode 


Figure  K.  31.  — Plot  of  Attack  Mode  Effect  by  Data  Set  and  Subject 
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Figure  E. :!(».  — Plot  of  Main  Effects  by  Subject  Pair  (Combined 

Data  Sets) 
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APPENDIX  F 

TRIAL  TWO  EXPERIMENTAL  DATA 
AND  ANALYSES 


In  this  appendix  the  following  abbreviations  apply: 
Threat  Perception  Model 


AT  - Acid  Test  Model 
1 - Model  One 

3 - Model  Three 

Attack  Mode 

HF  - Hover  Fire 
RF  - Running  Fire 


Exhibit  F.  1 

Summary  of  Critique  Responses 
This  exhibit  presents  summarized  responses  to  questions  comprising 
the  critique  administered  at  the  end  of  the  Phase  I Experiment.  Only  seven 
out  of  eight  subjects  completed  the  critique. 


I 

I 

I 


Subject  Number 


CRITIQUE 


The  questions  on  this  form  are  intended  to  stimulate  thought  concern- 
ing the  content  of  the  experiment.  Your  ideas  are  solicited  for  the  purpose  of 
improving  the  experiment  and  for  understanding  better  the  responses  you  gave 
during  the  experiment.  Please  feel  free  to  expand  on  your  answers  to  any  of  the 
questions  and  to  comment  on  any  other  topic  that  comes  to  mind.  Use  the  addi- 
tional paper  provided  if  you  need  more  space. 

I.  General 

A.  Was  the  intent  of  the  experiment  explained  satisfactorily? 

Yes  - 7 I would  enjoy  seeing  the  entire  program  in  operation 

No  - 0 when  it  is  complete  - 1 

B.  Were  there  times  during  the  experiment  at  which  you  felt  your  motiva- 
tion and  interest  were  lagging?  What  were  those  times? 

Yes  - 3 After  40  or  50  slides  I felt  my  interest  lagging  - 1 
No  - 4 

IL  Weapon  Information 

A.  Were  you  familiar  beforehand  with  the  friendly  and  enemy  weapons  dis- 
cussed in  the  experiment?  With  which  ones  were  you  least  familiar? 

Familiar  with  all  weapons  - 4 
Familiar  with  12.  5 and  37-1 
Slightly  familiar  with  14.  5-1 

Heard  of  all  weapons  but  no  personal  experience  with  any  - 1 

Least  familiar  with  Redeye  - 4 
Least  familiar  with  14.  5-1 
Least  familiar  with  23-1 

B.  Would  additional  information  concerning  the  weapons  be  beneficial? 
What  kind  of  information  would  be  most  helpful? 

Yes  - 3 Pictures  of  weapons  and  their  mountings  - 2 

No  - 4 Weapon  effectiveness  data  in  the  form  of  probability 

of  kill  as  a function  of  target  range  and  aspect  - 2 
Maximum  range  of  weapon  - 1 
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C.  Was  enough  time  allowed  for  studying  the  weapon  hand  outs? 

Yes  - 7 The  instructor  should  indicate  the  critical  data  that 
No  - 0 will  be  of  use  during  the  experiment  - 1 

D.  If  you  were  already  familiar  with  the  weapons,  were  the  modified  per- 
formance specifications  confusing  to  you? 

Yes  - 0 
No  - 7 

Did  you  use  the  information  as  presented? 

Yes  - 5 

No  - 2 I used  maximum  range  instead  of  maximum  effective 
range  - 1 

I allowed  for  pure  luck  in  5 second  exposure  situations 
(stated  reaction  time  was  8 seconds)  - 1 

E.  Was  the  choice  of  weapons  reasonable  and  representative  of  a mid- 
intensity environment? 

Yes  - 6 Some  other  types  of  weapons  will  probably  be  on 
the  battlefield  also  - 1 

The  VC  and  NVA  have  done  well  with  small  arms 
and  even  KPG's  - 1 

No  - 1 

IIL  Terrain  Information 


A.  Was  the  terrain  and  its  forested  areas  represented  with  sufficient 
clarity? 

Yes  - 7 
No  - 0 

Was  the  perspective  drawing  of  the  area  of  any  benefit? 

Yes  - 5 

No  - 2 I had  trouble  relating  the  drawing  to  the  map  - 1 
I did  not  look  at  the  drawing  - 1 
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B.  Was  the  line  of  sight  information  beneficial  or  did  it  introduce 
artificiality?  Would  it  be  of  benefit  to  have  this  information  in 
combat? 

Yes  - 7 The  information  can  be  obtained  by  a good  map  study  - 1 

No  - 0 The  information  is  not  available  in  combat  - 1 

I do  not  know  if  it  would  work  in  combat  - 1 

IV.  Threat  Evaluation 


A.  Did  you  have  difficulty  in  visualizing  the  situations  presented  to  you  in 
the  109  exposure  slides? 

Yes  - 2 I had  trouble  at  first  - 1 
No  - 5 

Did  you  have  particular  difficulty  in  assigning  a number  to  the  threat 
you  felt  in  the  ji'  nations  that  were  presented? 

Yes  - 2 The  threat  interval  (0-1000)  is  too  large.  A scale 
of  (1-10)  or  (1-100)  would  be  more  realistic  - 1 

No  - 5 

Were  some  situations  more  difficult  to  evaluate  than  others? 

Yes  - 5 I had  trouble  with  the  short  exposure  time  situations  - 1 
No  - 2 

B.  On  each  slide,  the  type  of  enemy  weapon,  the  exposure  time  and  the 
range  to  the  enemy  weapon  (maximum  and  minimum)  were  given.  In 
addition,  the  midpoint  range  was  called  out  verbally.  Were  all  these 
clues  of  benefit? 

Yes  - 7 
No  - 0 

What  other  factors  enter  into  your  perception  of  the  threat  posed  by  a 
solitary  enemy  weapon? 

Probability  of  kill  as  functions  of  range  and  attitude  - 1 
Luck  - 1 

Terrain,  density  of  foliage  - 1 

Aircraft  type  and  proficiency  of  crew  - 1 

There  are  so  many  variables  it's  a guessing  game  - 1 
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How  long  I am  in  his  field  of  fire  after  his  reaction  time  - 1 

None  - 1 

(Each  subject  had  a different  answer) 

C.  Would  you  say  that  the  threat  you  feel  from  the  14.  5 mm  and  the 

23  mm  weapons  is  almost  constant  out  to  their  respective  maximum 
ranges,  given  a constant  exposure  time  and  bearing  to  the  weapon? 

14.  5 mm  23  mm 

Yes  - 3 Yes  - 7 

No  - 4 No  - 0 

Would  you  say  that  these  weapons  are  more  threatening  in  your  rear 
quadrant  at  a given  range,  or  does  bearing  have  any  effect? 

More  threatening  from  rear  - 3 

More  threatening  from  side  - 3 

Makes  little  difference  - 1 

VI.  Route  Selection 

A.  Have  you  any  comments  concerning  the  situations  for  which  you  were  to 
select  a route?  Were  they  artificial? 

Yes  - 6 Some  cases  were  - 1 

Only  in  the  fact  that  the  exact  location  and  type  of 
weapon  was  given  - 2 

In  combat,  line  of  sight  information  is  not  available  - 1 
I would  not  attack  a weapon  that  had  a greater  effective 
range  than  I;  there  is  a thin  line  between  mission 
accomplishment  and  suicide  - 1 

No  - 1 

B.  Were  you  permitted  sufficient  latitude  in  the  tactics  to  be  used  during 
the  attack? 

Yes  - 6 Not  enough  is  known  about  tactics  in  this  type  of 

No  - 0 environment  - 1 

C.  In  general,  can  you  describe  the  differences  between  two  attack 
situations  that  would  lead  you  to  select  hovering  fire  in  one  and  running 
fire  in  the  other? 
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-Running  fire  is  a method  of  attack  used  during  periods  of  high 
density  conditions,  or  when  there  is  a lack  of  suitable  terrain  to 
allow  engagement  at  maximum  effective  range  of  the  weapon 
system. 

-If  I were  attacking  a target  inside  that  weaporis  range,  I would 
prefer  to  be  moving  when  I am  attacking.  If  I am  outside  that 
weaporis  effective  range  or  I feel  he  has  a relatively  slight 
chance  of  hitting  me,  then  I would  fire  from  the  hovering  position. 

-It  would  depend  on  how  close  I could  get  to  the  target  without 
being  seen. 

-I  would  use  hovering  fire  only  if  I could  locate  a firing  position 
that  has  high  vegetation  or  terrain  behind  it  so  I would  not  be 
silhouetted  against  the  sky. 

-I  cannot  set  up  conditions  for  hovering  versus  running  fire  because 
I've  never  (in  combat)  seen  a need  for  it.  (Times  change) 

-My  answers  were  explained  on  the  Master  Information  Form. 

-Hovering  fire  is  better  if  cover  is  available  and  your  max  effective 
range  is  greater  than  that  of  the  enemy.  When  you  must  expose 
yourself  inside  the  enemy's  range,  running  fire  is  better. 

D.  In  selecting  a launch  range,  did  you  trade  off  kill  probability  against 
exposure  time? 

Yes  - 5 
No  - 2 

What  criterion  did  you  use  in  arriving  at  a launch  range? 

-Terrain  was  the  major  criterion;  I tried  to  engage  at  maximum 
effective  range. 

-I  would  rather  take  a few  more  chances  of  being  hit  than  miss  the 
target  and  have  to  engage  again. 

-I  tried  to  maximize  kill  probability  while  exposing  myself  for  less 
time  than  the  reaction  time  of  the  enemy. 

-I  considered  the  visibility,  effective  range  and  time  to  react  of 
the  enemy. 
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-I  tried  to  minimize  my  exposure  inside  the  enemy's  effective 
range. 


E.  What  would  best  describe  the  way  you  selected  routes?  Did  you 
minimize  travel  time  or  exposure? 

Exposure  - 7 Minimum  exposure  is  critical  - 2 

Travel  time  - 0 Travel  time  is  immaterial  - 2 

Did  you  maximize  the  element  of  surprise? 

Yes  (attempted  to)  - 7 Surprise  is  nice  but  not  necessary  - 1 
No  - 0 Minimum  exposure  produces  surprise  - 1 

Did  you  try  to  remain  out  of  the  effective  range  of  the  enemy  weapons? 

Minimum  exposure  is  critical  even 
beyond  maximum  effective  range  of 
23  mm  - 1 

23  mm  had  enough  range  to  prevent 
this  tactic  - 2 

■ 

VL  Other 

The  information  you  get  from  us  may  not  be  very  valid.  Not  enough  is 
known  about  mid-intensity  employment  of  helicopters.  More  development, 
other  than  yours,  is  needed  - 1 


Yes  (attempted  to)  - (> 
No  - 0 


I 

I 


Table  F.  1 


1 


Alternative  Numbering  Scheme 
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Table  F.  2 


Summary  of  Similarity  in  Routes  Produced  by  the 
Three  Threat  Perception  Models 


Between  Model  One 
and  Model  Three 


RF 

HF 

Similarity 

<%) 

Totals 

<%) 

0 

0 

0 

0 

1 

50 

25 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 8 4 


1 - route  produced  by  one  of  the  two  models  is  the  same  as  that  produced 
by  the  other 

0 - different  routes  are  produced 
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Table  F.  3 

Summary  of  Similarity  in  Route  Alternatives 
Presented  to  Comparable  Subjects 


- 1 - at  least  one  route  alternative  of  paired  subject  is  the  same 
as  the  alternative  in  question 

0 - no  comparable  route  alternative  exists  for  paired  subject 


Table  F.  4 


0.05  0.05  0.  85 


Table  F.  5 (Continued) 


A 


*R  - rating 

A - time  exposed  during  attack  (seconds) 

1 - time  exposed  to  target  weapon  only  (seconds) 

2 - time  exposed  to  supporting  weapon  only  (seconds) 

3 - time  exposed  to  both  weapons  simultaneously  (seconds) 
T - time  exposed  to  one  or  more  weapons  (seconds) 

458 





Table  F.  6 


Exhibit  F.  2 


Trial  Two  Experiment  Mailer 

This  exhibit  represents  a sample  of  the  mailer  distributed  to  each 
subject  during  the  Trial  Two  Experiment.  It  is  actually  a copy  of  the  mailer 
received  by  subject  7 and  contains  the  responses  he  prepared. 
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THU  OHIO  S'I'AIK  UNIMOKSITV 

28  February  1972 
MEMO: 

(Names  omitted  for  anonymity) 

FROM:  Don  C.  Hutcherson 

During  our  last  session  together  several  subjects  had  very  constructive  com- 
ments regarding  undesirable  characteristics  of  the  attack  routes  selected 
through  computer  analysis.  The  fact  that  the  route  alternatives  were  not  as 
good  as  desired  was  subsequently  revealed  in  the  rating  data  that  were  collected. 

Based  on  comments  received,  the  computer  model  has  been  modified  and  the 
analyses  have  been  repeated.  However,  the  number  of  subjects  has  been  re- 
duced to  six,  only  alternatives  for  subject  routes  one  and  three  have  been  pre- 
pared, and  the  number  of  alternatives  for  each  route  has  been  reduced. 

I am  asking  that  you  provide  me  with  comparison  data  for  the  routes  prepared 
by  the  new  computer  model.  Your  familiarity  with  the  problem  coupled  with  the 
reduced  amount  of  analysis  required  should  permit  you  to  complete  the  work  in 
thirty  (30)  minutes  at  the  most. 

Please  take  a few  minutes  and  provide  me  with  the  desired  information  at  your 
earliest  opportunity.  In  order  to  meet  my  deadlines  here,  I must  have  your 
response  in  my  possession  by  8 March  1972.  Your  cooperation  is  appreciated 
by  me  personally  and  by  Systems  Analysis  Group,  Combat  Developments 
Command. 


Instructions 


Read  the  attached  description  of  enclosures. 

Carefully  examine  the  enclosures  for  understanding. 

Perform  the  comparisons  in  a fashion  identical  to  that  used  during  our 
last  session: 

a)  If  an  alternative  is  definitely  the  same  in  value  as  your  route,  rate 
it  100  in  column  five  and  check  the  first  column  of  the  answer  sheet. 

b)  If  you  are  almost  indifferent,  rate  the  "non-preferred"  route  between 
95  and  100  in  column  five  and  enter  "my  route"  or  "computer  route" 
in  column  two  to  indicate  which  one  is  preferred. 

c)  If  you  mildly  prefer  one  route  to  another,  rate  the  "non-preferred 
route  between  70  and  95  in  column  five  and  enter  "my  route"  or 
"computer  route"  in  column  three  to  indicate  which  one  is  preferred. 

d)  If  you  strongly  prefer  one  route  to  another,  rate  the  "non-preferred" 
route  between  0 and  70  in  column  five  and  enter  "my  route"  or 
"computer  route"  in  column  four  to  indicate  which  one  is  preferred. 

As  before,  the  alternatives  are  sequentially  numbered  and  there  may  be  a 
different  number  of  alternatives  to  route  one  than  there  are  to  route  three. 

A maximum  of  six  alternatives  can  exist. 

The  computer  plot  of  your  route  is  included  for  information  only.  Naturally, 
it  is  not  to  be  rated  as  it  represents  your  route.  It  was  taken  from  the 
original  acetate  tracing. 

Mail  the  entire  package  in  the  postage  free  envelope  provided.  I should 
receive  your  response  by  8 March  1972. 


A 


Description  of  Enclosures 

You  will  find  the  following  material  enclosed  for  each  of  your  routes  (routes 
1 and  3). 

1.  The  contour  map  upon  which  you  transferred  your  route  during 
our  last  session. 

2.  A plot  of  your  route  prepared  by  the  computer. 

3.  A statistic  sheet  giving  basic  information  about  your  route  as 
determined  by  the  computer. 

4.  A plot  of  alternatives  to  your  route  as  prepared  by  the  computer. 

5.  Statistic  sheets  giving  basic  information  about  each  alternative 
as  determined  by  the  computer. 

6.  An  answer  sheet  to  record  your  comparison  information.  (Any 
additional  comments  are  appreciated  and  can  be  entered  on  the 
answer  sheet. ) 

Items  2,  3,  and  5 are  new  and  are  enclosed  to  assist  you  in  your  evaluation  of 
alternatives.  They  give  detailed  information  that  was  not  available  during  the 
last  session.  Such  information  may  be  especially  valuable  in  judging  your  own 
route  in  relation  to  the  alternatives. 
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<c**rV  p RAIL  RCUTE  STATISTICS*** 


TOTAL  ROUTT  LFMGTH 20834.  METERS 

TOTAL  TRAVEL  TIME  730.  SECONDS 

TOTAL  EXPOSURE  TIME: 

TO  TARGET  WEAPON  ONLY 0.  SECONDS 

TC  SUPPORTING  WEAPON  ONLY  ll<5.  SECONDS 

TO  ROTH  UP  A PC  NS  SIMULTANEOUSLY  - 98.  SECONDS 

***FIRINC  STATIST!CS*** 

TYPE  OF  FIRE  HCVER  FIRE 

ALTITUDE  OF  TERRAIN  AT  TARGET 349.  METERS 

RANGE  TC  TARGET  AT  LAUNCH 2541  . M FT  FRS 

LAUNCH  ALTITUDE  (!)  215.  METERS 

MISSILF  TIMF  OF  FLIGHT  13,  SECONDS 

RANGE  TC  TARGET  AT  RESILE  1 m PACT  — 2541.  METERS 

ALTITIJCE  AT  MISSILE  IMPACT  (1) 215.  METERS 

DIVE  ANCLE  CF  MISSILE  (2  1 6.  DFGREE  S 

FXPOSURE  TIME  FOR  FIRING  EVRNT  (3)  - 9R.  SECONDS 

PROPAPILITY  OF  KILL 


ff $*** 

(1  ) HFL  ICOPTER  ALTITUDE  ABOVE  TEPRAIN  (OR  TREE  TOPS 
IF  ABOVE  WOCDLT  AREA). 

(2)  ALSO  CORRESPONDS  TP  HELICCPTFR  DIVE  ANOLE  FCR 
PUNNING  FIRE.  NEGATIVE  VALUE  INO  IC  A TE  S C L I MB. 
(31  INCLUDES  C L I M P TO  FIRING  PCSITION,  AIMING  CELAY 
FLIGHT  OR  MISSILE  AND  OF  SC  FNT  DURING  RRFAKAWAY. 


AD-A045  094 


UNCLASSIFIED 


AD 

A045094 


OHIO  STATETuNIV  COLUMBUS  SYSTEMS  RESEARCH  GROUP  F/G  1 

THE  HELICOPTER  ROUTE  SELECTION  MODEL  (DYNFLITE) . (U) 

AUG  76  G M CLARK*  D C HUTCHERSON*  D C BITTERS  DAAG25-70-C-0311 
RF-2978-FR-76-4(0)  NL 


Original  size  of  map:  7.5  x 14.  5 inches 


size 


S'JDJFCT 


r cute  1 running  tire 


ALT  1 


***CVERALL  ROUTE  ST  AT  I S TI CS*** 


TOTAL  ROUTE  LENGTH 12J04.  Ml  TEES 

TOTAL  TRAVEL  T[mr  ? 9 7 • SECONDS 

TOTAL  EXPOSURE  T I ME : 

TO  T AR GF"”  WEAPOV  ONLY 0.  SECONDS 

TO  SUPPORTING  WEAPON  ONLY  56.  SECONDS 

TC  BOTH  WEAPONS  SIMULTANEOUSLY  - 3<~‘.  SLCONDS 


<**FJR  INC,  STATIST[CS*** 


TYPE  OF  EIRE  RUNNING  F I°F 

ALTITUDE  OF  TERRAIN  AT  TAPGFT ?A.o.  METERS 

RANGC  TO  TARGET  AT  LAUNCH 2100.  METERS 

LAUNCH  ALTITUCC  111  1°.  METERS 

MISSILE  TIME  OF  FI  1 0 H T 10.  SECONDS 

RANGE  TO  TARGET  AT  MISSILE  IMPACT  — 16PR.  METfFS 

ALTITUDE  AT  MISS!LC  IMPACT  (1)  1°.  MF.TEES 

DIVE  ANGIE  nr  MISSILE  (?)  0.  DEGREES 

E X°OSUR F TIME  ^OR  FIRING  EVCNT  (?)  - 15.  SECONDS 

PROBABILITY  OF  KILL  7*f 


***HjQT  ES** 


(1)  HELICOPTER  ALTITUDE  ARCVE  TERRAIN  (CP  TREE  T^PS 
IF  ABOVE  WOODED  AREA). 

(?)  ALSO  CORRESPONDS  TO  HELICOPTER  DIVE  ANCLE  FOR 
RUNNING  FIFE.  NEGATIVE  VAl.UF  INDIC ATES  CLIMB, 
(3)  INCLUDES  C L I M B TO  TIRING  POSITION*  AIMING  DELAY, 
FLIGHT  OF  MISSILE  AND  DESCFNT  DUPING  BREAKAWAY. 


L a 


r>f.i 


fT  F * f 


p , f B 


r »**•  ' 
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ALT  2 

***PVEB  ALL  pn.jTE  STATISTICS*** 


T'->7AL  P OlJT  F LENGTH ?15«3.  MtTEPS 

T r ’ T A L T 9 A V p L T I Mr c55.  SECONDS 

TOTAL  EXPOSURE  TI*'E: 

to  TAPGFT  WEAPON  ONLY <)  . SECOND 

Tr  SUPPORTING  WEAPON  ONLY  40.  SECGNDS 

rn  BOTH  WE  APONS  S I YUL T AMP TJ SI  Y «-  10.  SECONDS 

* * * F IP ING  STATISTICS*** 

TYPE  CF  F I b E RUNNING  FIRE 

A LT ! TIJOE  OF  tpi^p  aim  at  TARG£T ?4C.  MFTTRS 

RANGE  TO  TARGET  AT  LAUNCH 4 - S . METERS 

LAUNCH  ALTITUDE  ID  ?7 . Mf  T c R S 

MISSILE  T I M c CF  FLIGHT 2 . SECDNOS 

RANG1*  TO  TiRGFT  AT  MISSILC  !MPiCT  --  ‘rr-'.  mct^RS 

ALTI^UnE  AT  YissiLC-  IMPACT  (i)  10.  M7TE RS 

LI7F  ANGLE  CF  YISSTL'7  (?)  F • DEGREE  S 

FXPQSURE  TIN?  FOR  e I R ING  EVENT  ID  - 10.  SECONDS 

PROBABILITY  OF  KILL  .11- 


***NQTES*** 

( I ) HPL I CO  P TFF  ALTITUCF  ABOVc  TFRRAIN  (C»  TRFE  TOPS 
IF  ABOVE  WOODED  AREA),, 

(D  ALSO  C 0»Pf SPCNDS  T0  HELICOPTER  CIVE  ANGLE  FOR 
RUNNING  THE.  NEGATIVE  VALUE  INDICATES  C L I M 3 • 
li)  INCLUDES  CLIMB  TC  F I D I N G POSITION,  AIMING  DELAY, 
FLIGHr  0 r-  MISSILE  AND  HE  SCENT  DOPING  BREAKAWAY. 
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SUB  JSC  T 7 P <~UTS  ' PUNNING  FIRE 


ALT  3 

***OVEP ALL  ROUTE  STATISTICS*** 


TOTAL  ROUTT  LENGTH 12310.  METERS 

TOTAL  TRAVEL  TIME ^0.  SECCNPS 

TO” AL  EXPOSURE  TIME: 

T0  '•’ARGET  WEAPON  ONLY 0.  SFCGNPS 

TCi  SUPPORTING  WEAPON  ONLY  57.  SECONDS 

TO  ROTH  WEAPONS  SIMULTANEOUSLY  - 64.  SFCONDS 

* '■  * F 1 p I N G STATISTICS*** 

TYPE  CF  EIRE FUNNING  FIRE 

ALTITUDE  OF  t E-'  R P A I N AT  TAFGE-r 'AS.  MITERS 

RANGE  TO  TARGET  AT  LAUNCH  2500.  METERS 

LAUNCH  Al T I TUf‘E  (?)  IF.  METERS 

MISSILE  TIME  FliO-HT 1?.  SECONDS 

RANGE  TO  TARGET  AT  MISMLE  IMPACT  — 201?.  vpTEPS 

ALTITUDE  at  MISSILE  IMPACT  (I) IP.  METERS 

DIVE  ANGLE  OK  ‘'ISSUE  (?)  !.  DEGREES 

EXPOSURE  TIME  FOR  FIRING  EVENT  (3)  - ]?.  SECONDS 

PROBABILITY  nr  KILL  / s 


***f!OTFS*v* 


m HEL  ICOP  TF  p ALTITUOC  ABOVE  Terrain  (CP  TRIE  TOPS 
IF  ABOVE  WO OP  FT  A^fa). 

(?)  ALSO  CORRESPONDS  ^ HEL ICOPTFR  DIVE  ANGLE  FOR 
RUNNING  FIRE.  NEGATIVE  VALUE  INDICATES  CLIMB. 
(3)  INC  LUC l S CLIMB  tC  FIRING  POSITION,  AIMING  PELAY, 
FLIGHT  OF  MISS' IF  AND  DESCENT  PUP  IMG  BREAKAWAY. 


SUBJECT  7 


»CUTp  » HOVER  F 1°  R 


ALT  4 

< V* OVER ALL  P«  C U T t STATISTICS*** 


TOTAL  route  length 


'SEE?.  METERS 


T 1 T A L T RAV^l  time  

TUT  AL  cXPO  SUP  r TIME: 

TO  T A® GFT  WEAPON  ONLY 

Tj  SUPPORTING  WEAPON  ONLY 

to  ofTH  WLAPC^'S  SIMULTANEOUSLY  - 


* F " . SECONDS 


0. 

1 27. 

c ? _ 


SECONDS 

SECONDS 

SECONDS 


v*-*r  jp  imp,  STATISTICS*1**' 

TYPE  OF  FIRE  

ALTITUDE  OF  TERRAIN  AT  TARGET 

RANC.F  TfJ  TARGET  AT  LAUNCH  

L AUNfH  ALT  I TUDF  ( 1 ) 

MSS’LC  TIME  OF  FLIGHT 

RANGE  n TARGET  at  MJSMLF  IMPACT  — 

ALTITUDE  AT  MISSILE  IMPACT  (1)  

OIVc  ANGLE  °F  m I s R r l f.  (•>)  

EX  Popup  E TIME  FC'F  FIRING  cVENt  (R)  - 
PR-iraPTLTTY  ~F  KILL  

NCTrS*** 


(•)  McLICrTT.  F ALTTTlJI<F  ABOVE  TERRAIN  (OR  TREF  ~npp 
IF  A RL'VF  V^TD  Ff  • A R r A ) • 

(?)  ALSO  COPt  r.R PONDS  Tf  HELICOPTER  DIVE  ANGLE  FOR 
RUNNING  FIRE.  NEGATIVE  VALUE  INDICA^t, 

(?)  TNCLUrtS  C L T M P TC  FIRING  PncITION,  AIMING  DELAY, 
(LIGHT  rP  MISSILE  4NC  DESCENT  DURING  PRE AKAWAY. 


HCVFP 

FIRE 

. 

MF  Tr  0 S 

2 ECO  . 

yr  ’•f  p c 

20  7 . 

Mr  rrps 

i:. 

seconds 

2E00. 

VF  T r R S 

207. 

MET  F KS 

* . 

D f G R r t c 

S2. 

SECONDS 

-H  /*  'T  ^ 

Or  A IN-- 


J 


SURJFCT  7 PC'UTC  l H OVER  UPC 


ALT  5 

**«CVF!RALL  ROUTE;  S.T  A T I ST  I CS**  * 


TOTAL  ROUT  r LENGTH .136!  6.  VETERS 

TOTAL  TRAVEL  TIME 626.  Sf  CONES 

TOTAL  EXPOSURE  T!Mf.  : 

TO  TARGET  WEAPON  ONLY 0.  SECONDS 

rc  SUPPORTING  WEAPON  ONLY  0.  SECONDS 

to  HPTI-*  WEAPONS  SIMULTANEOUSLY  - ! 3 . SECONDS 


***FIRING  STATISTICS *** 


TYPE  Or  PIPE liCVER  EIRE 

ALTITUDE  CF  TERPMN  AT  TARGET  ?6C.  METfPS 

RANGE  TO  TAPGET  t T LAUNCH T^OO.  M£TrkS 

LAUNCH  ALT ! TDD = (11  lp.  METERS 

NISSTLP  TINE  CF  c L I GHT  9 . SECONDS 

RANG."  TO  URGE!  AT  MISSILE  IMPACT  — 1700.  MPTFRS 

ALT ITUDE  AT  MISSILE  IMPACT  (1)  19.  VOTERS 

DIVE  ANGLE  OF  MissiLc  12)  0.  DEGR^cs 

EXPOSURE  TIME  FOR  TIRING  EVENT  (3)  - 17.  SECONDS 

PRnpjpHiTy  ->F  KUL  .7 % 


*+*NCTES*T* 

(1)  H£L  ICOP  Tr  R A L T T T U 0 " ARC VP  TERRAIN  (CD  T R r E TOPS 
IE  ABOVE  Knrr  ED  AREA). 

(?)  ALSO  C ?RR  F SP( NOS  TD  HELICOPTER  DIVF  ANGLE  PQR 
RUNNING  F!Rf.  NEGATIVE  VALUE  INDICATES  CLIMB. 


( 7 | INCLUDES  CLIVE  TO  c I R I N G POSITION*  AIMING  OfcLAY, 
FLIGHT  OF  VISSILF  AND  DESCENT  DUP I NG  BREAKAWAY. 


S 7 R ? 


own  CHPirr 


***0VPRALL  ROUTE  statistics*** 


TOTAL  ROUTE  LENGTH 20051.  ME  TER  S 

TOTAL  TRAVEL  TIMF  690.  SECONDS 

TOTAL  EXPOSURE  TIME: 

TO  TARGET  WEAPON  CNLY  75.  SECONDS 

to  supporting  weapon  only  o.  seconds 

TC  POTH  WEAPONS  SIMULTANEOUSLY  - 52.  SECONDS 

***FIRING  STATISTICS*** 

TYPF  OF  FIRF  HOVER  F IR  F 

ALTITUDE  OF  TERRAIN  AT  TARGET 386.  METERS 

RANGE  TC  TARGET  AT  LAUNCH 2011.  METERS 

LAUNCH  ALTITUDE  ( ! ) 189,  METERS 

MISSILE  TIME  TF  Ft.  IGHT 10.  SECONDS 

RANGE  TC  TARGET  AT  MISSILE  IMPACT  --  2011.  METERS 

ALTITIJOE  AT  mtsSILE  [ MFAC  T (11  189.  METERS 

DIVE  ANGLE  OF  MISSTLC  (2)  5.  DEGREES 

EXPOSURE  T I vf  F C P FIRING  EVENT  ( 3 » - 82.  SECONDS 

PRPPAPILITY  OP  KILL ,1  ^ 


* **NOT  FS  * * * 


(1)  Hc  l.IC  CPTE  R ALTITUCE  APOVF  TERRAIN  (CR  T PFF  TOPS 
IF  APOVF  WOODED  AREA). 

(2)  ALSC  FORRES  PONGS  TO  HFLICCPTER  DIVE  ANGLE  FOR 
RUNNING  FIRE.  NEGATIVE  VALUE  INDICATES  CLIMB. 

(3  > INCLUDES  CLIMB  TO  F 1°  I NG  POSITION,  AIMING  DELAY, 
FLIGHT  OF  MISSILE  Anc  CFSCENT  CURING  BREAKAWAY. 


size 


9 l .•  B J t T 7 i-CMiTF  A RUNNING  F J P £ 


ALT  1 

T*”-PvF  RALL  CHUTE  STATISTICS*** 


TH T AL  ROUTE  IFNGTH 1<?R£5.  MITERS 

TOTAL  TRAVEL  TTMF c9i  . Sf-CONC'S 

TOTAL  rXPUSUR c TIM*: 

TP  TAR  G -T  WEAPON  ONL  Y 25  1 . SF  CONE  S 

TO  ? UP PC’R ’■  T NG  WEAPON  Oh. L Y 0.  SFCCNOS 

TP  BOH  WEAPONS  SIMULTANEOUSLY  - i<J.  SECONDS 


* FIRING  STAX  IS  TTCS"** 


TYPE  OP  EIRE P.IJN  hi  IMG  Pike 

A l.  T I T U 0 P OF  T-RRAtN  AT  TARGE” -F6.  MfTfRS 

RANGE  TO  TARGET  at  LAUNCH BOO.  METERS 

LAUNCH  ALTITUDE  ( 1 ) ?F.  voters 

"ISSIL:  TI  ME  OF  PLIGHT E . SfCONHS 

PANGr  *0  TARGET  AT  MT  S^I  LF  IMPACT  — -0?.  MfTERS 

ALTITUDE  AT  MISSILE  IMPACT  (I) 10.  METERS 

r i vs  angle  of  ^iss;ir  m i. . ouc,pr.rz 

EXPOSURE  TIME  F0°  c I R IMG  EVENT  (-)  - 10.  SlC°NHS 

PRGRAblL  [TV  OF  KILL  9 Z, 


* * M N 0 T F S***  * 

(.’.  ) MEL  I CO  p TPR  ALTTTUOF  ABOVE  TERRAIN  (OR  TkTE  TOP  $ 
IP  ABOVE  WOO BE  0 AR  c A ) . 

( 2 ) ALSO  C H°R F S PONDS  TP  HEL  IPHPTER  DIVE  AN GLF  FOR 
RUNNING  f-IRF.  NEGATIVE  VALUE  INDICATFS  CLIMB. 

( 1 ) INCLUDES  CLIMB  Tp  FIRING  POSITION,  AIMING  HEL  \Y  • 
FLIGHT  nc  MISSILE  AND  DpS  CF:NT  DUPING  BREAKAWAY. 


SUBJECT  7 


RPUTC  3 RUNNING  FIRE 


ALT  2 


***OVERALL  ROUTE  STATISTICS'''** 


TOTAL  ROUTE  LENGtH 2107R.  METERS 

T0TAL  tpavEL  TIME 63'?.  SECONDS 

TOTAL  EXPOSURE  T ! RIF  : 

TO  TARGET  WEAPON  ONLY *,0.  SECONDS 

TO  SUPPORTING  WEAPON  ONLY  0.  SECONDS 

TO  ROTH  WEAPONS  SIMULTANEOUSLY  - 10.  SECONDS 


***ETPING  STATISTICS*** 


T YnE  OF  FIRE RUNNING  EIRE 

ALTITUDE  Or  TFPRATM  at  TARGET 266.  METEES 

RANGE  TO  TARGET  A 7 LAUNCH EDO.  METERS 

LAUNCH  ALTITUDE  (1)  26.  METERS 

MISSILE  TIME  OF  FLIGHT  2.  SECONDS 

RANGE  TO  T A R G T AT  MISSILE  JMPACT  — 602.  METRPS 

ALTITUDE  AT  MISSILE  IMPACT  (1)  10.  METERS 

DIVE  ANGLE  0^  missILE  (?) !.  DEGREES 

EXPOSURE  TIME  FOR  FIRING  EVENT  (3)  - ]0.  SECONDS 

PROBABILITY  OF  KILL  ^ ■{, 


*** NOTES*** 

(II  HELICOPTER  ALTrT(tCE  ABOVE  TERRAIN  (OR  TREE  TOPS 
IF  ABOVE  WOQDr,j  AREA). 

(2)  ALSO  CORRESPONDS  HELICOPTER  DIVE  ANGLE  FOR 
PUNNING  FIRE.  NEGATIVE  VALUE  INDICATES  CLIMP. 
("*)  INCLUDES  CLIMB  TO  FIRING  POSITION,  AIMING  DELAY, 
FLIGHT  OF  MISSILE  AND  DESCENT  DUPING  BREAKAWAY. 


SUB JFO  T 7 


RO'JT“ 


RUNNING  -IRS 


ALT  3 

* * * C V E 0 A L L ROUTE  STATISTICS*** 


TOTAL  ROUTE  LENGTH ZO^Ol.  MTTF  R S 

TOTAL  TRAVEL  Tl^t  t!f.  SECONDS 

TOTAL  exposure  time: 

TO  ’’A'>G£T  WEAPON  ONLY ’17.  SECONDS 

TQ  SUPPORTING  WEAPON  ONLY  0.  SECONDS 

TO  BOTH  W'-APCNS  $ I MUL TA MSOUS LY  - 3 77  . SECONDS 

♦** FIRING  STATISTICS*** 

T Y ° £ Q'  PIPE  RUNNING  PIPE 

ALTITUDE  OF  TERRAIN  AT  TARGET  ??‘.  MCTEPS 

RANGF  tc  TARGET  AT  LAUNCH £95.  METERS 

LAUNCH  ALTITUDE  (1)  2?.  METERS 

MISSILE  TIME  OF  FLIGHT A.  SECONDS 

RANGE  TO  T AR  f,L r AT  MIS'ILE  IMPACT  — 7?0.  METRES 

ALTITUDE  AT  MS5U-  IMPACT  (!) 21  . METERS 

LIVE  ANGLE  OF  mISSILF  (2)  DEGREES 

EXPOSURE  T I Mc  FOR  FIRING  FVENT  (0)  - M . SECONDS 
PROBABILITY  OF  KILL  


***NOT nS*+* 

(1)  H E L I CO 0 T 1 c ALTITIJDF  APOV"  TERRAIN  (OR  TRE  F Tr;pc 
IF  A3GVF  W00r£r  A 0 r A ) . 

(?)  ALSO  CORi-f  SPENDS  To  HEt  IC’TOTER  D!Vr  ANCLE  FOR 
RUNNING  'of.  NEGATIVE  VALUE  INDICATES  CLI^d. 

( T ) INCLUDES  CLIMB  rC  FIRING  POSITION.  AIMING  DEI  AY. 
FLIGHT  OF  M I c c £ l r AND  OESCEN"  DURING  BREAKAWAY. 


nr.q 

Dt-H 


r * H 


* } 

n*'1 


* T~!r 


f*.  !>!  ( 
M 


482 


S'lftJFC.T  7 


P CUTE  3 HOVER  cn? 


ALT  4 


*'**CVERA  LL  ft  OUT  E ST  AT  1ST  I CS*^  * 


TOTAL  ROUT?  LENGTH 1533*.  ^’TERS 

TOTAL  r RAVEL  ttv? 486.  SECONDS 

TOtAL  EXPOSURE  TIME: 

TO  TARGET  WEAPON  ONLY 21!.  SECONDS 

7 C SUPPORTING  HEATON  ONLY 0.  SECONDS 

TC  «TTH  WEAPONS  SIMULTANEOUSLY  - 23.  SECONDS 


***FTRING  STATISTICS*** 


TY°E  OE  r I ° E HCVFP  EIRE 

ALTITUDE  0 E "STRAIN  AT  TARGET ER6.  >*»-'  TE  R S 

PANGF  TC  TARGET  AT  LAUNCH  2500.  METPRS 

LAUNCH  ALTITUDE  ( 1 I 2'*.  METERS 

WTSSILE  TIME  OF  FLIGHT 13.  SECONDS 

RANGE  TO  T A R G R T AT  FISSILE  IMPACT  — ? *500 . METF&S 

ALTITUDE  AT  'MSS!L‘:  I MPAC r C) 20.  METERS 

riVF  ANGLE  OF  MI$SILC  ( E ) 1.  AGREES 

EXPOSURE  TIME  FOR  FIRING  EVENT  13)  - 22.  SECONDS 

PPQRARTL  I f Y OF  KILL  .i5 


***NOTES * '** 


(I)  HELICOPTER  A L T I T !J  D " AGCV'"-  TERRAIN  <0=  T'U-C  TCFS 
TF  AP-'V-  WOODED  AREA)  . 

(> ) ALSU  COROFSPGEC;  r0  HELICOPTER  CIVT  ANGLE  FOR 
RUNNING  FlRf.  NEGMlVc  VAL'JE  INDICATES  CLIMR. 

( 3 ) INCLUDES  C L I v 3 TO  FIRING  POSITION,  AIMING  DELAY  , 
rL I GHT  QP  MISSIL"  AN C DESCENT  DUPINC  BREAKAWAY. 


SUBJECT  7 ROUTS  ? HfVFR  FJPF 


ALT  5 

;t ny  ER*  LL  ROUTE  57  AT  ! ST  ICS*** 


'rOT AL  I ENGTH ?o?i«.  METERS 

Tn‘r  AL  TPAV-L  TIMc ?Or-.  SECONDS 

TOTAL  EXPOSURE  TIMS: 

in  T AH  G F T WEAPON  PNIV 40.  SFCONDS 

TO  SUPPOSING  WEAPON  ONLY 0.  SECONDS 

TC  F*CTH  W- A PuflS  S J MULT  AN"  OUSLY  - - . S -T.CNCS 


♦ -♦FIRING  STATISTIC S**  * 


ryoc  cj^c MQV  [«  c J p c 

ALTITUDE  r'n  A ""  TARGET -><16.  ME  TF  P S 

PANGS  TO  TAf?G:CT  AT  LAUNCH 900.  V’-TFPS 

LAUNCH  A|.T  I TUO'*  (II  H.  M ~ T F P 5 

M ! 5 S I L r T I M F OF  c L I GHT 4.  SECONDS 

RANG*-  TO  tAwCFT  at  R 1 S S I L c IMPACT  — QOO . METERS 

ALTITUDE  AT  *!SSILE  IMPACT  (11  13.  METERS 

OIVc  ANGIE  OF  *-i  I s S 1 1.  E < ? > -I.  OEGP  F^S 

FXPOSUFE  T T ME  -OR  FIRING  EV-NT  (})  - D.  SECONDS 

pcOPipiL  [TY  0 = KILL  .iC 


^♦NO^ES*** 

(1)  HFI.  ICGPTf-p  ALTITUTF  AROVc  TERRAIN  (OP  t*0E  TQPC 
IF  ARC  VF;  WHGOFO  A Pc  A ) . 

( ’ I ALSO  CORRESPONDS  Tn  HELICOPTER  DT V F ANGLE  Ffr 
RUNNING  rpc#  fjr  Q AT  I v ' VALUE  INDICATES  CL  I MR  . 
(M  IMCLUIJPS  CLIMB  Tn  FIRING  POSITION,  AIMING  DELAY, 
FLIGHT  oc  MISSTLF  AND  DESCENT  DURING  BREAKAWAY. 


f ’ ' * ! 


r-  *>,»  f 
1 
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ALT  6 

v**  ryf-R  AL  L *5 TUT E ST  AT  T ST  I CS**  * 


T n f A L cn'JTE  LFNGTH 1*^356  . M r T F R S 

DU  7 Al  TRAVEL  ’1Vr  51?.  SECONDS 

TOTAL  EXPOSURE  Tl^': 

TAPGRT  WFAP^M  ONLY ’17.  SECONDS 

T Q SUPPORTING  W c A PO  M ONLY 0.  SECONDS 

rr  ao:h  wriP0f'S  SIMULTANEOUSLY  - 178.  SECONDS 


**•*  F IP  INC  STATIST  ICS  *** 


t yp£  Oc  f I P r-  I'CVC  r r I R C 

ALTITUDE  nr  T CP»  A 1^  at  tar  GF  r sph.  MiTcRS 

RANGE  TO  TARGET  AT  LAUNCH 100.  METERS 

LAUNCH  AI.t  * TUPP  (I)  75.  MrTrss 

MISSILE  Ti^L  Ce  FLIGHT  u,  SrT  CND  S 

RANG"  TO  TARGET  AT  M!  SSIlf  I «s>\CT  — '‘00.  MLTT  c S 

ALTT  Tljnc  A r m J 5 S I L :3  :*'RACT  I”) 2 5.  m:tckS 

GIVE  ANGLF  OF  MTSSILF  (?)  -•.  DcGPFFS 

EXPOSURE  TIME  CCP  FIRING  2VL  ’JT  (?)  - ’ D . SECCNnS 

PR  TP  AP?  L IT  Y nr  KIU.  ^ 


***i\L’T  f.  S*** 

(•)  HTIO'PTfk'  ALTITUDE.  Af^Vc  T E L S A I N (CP  T RR  r T-jPS 
IF  A30VE  WOODED  A - ; A ) . 

C’)  ALS°  CORRESPONDS  T HELICOPTER  D!Vc  ANCLE  FTP 
RUNNING  FIRF.  NEGATIVE  VALUE  INDICATES  rLIMp. 
(?)  INCLUDES  CLIMp  TO  FIFING  P^SITI™'.  AIMING  PctAYf 
FLIGHT  OF  MISSILE  AND  DESCENT  DUPING  pcEAMwAY. 


p?  rf"j  * \ i * t 5 f 
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J 


Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares 

Significance 

1 -Block 

1 

12695.96 

12695.96 

o|  + 12  of 

♦ ** 

2-  Route 

1 

6800. 65 

6800.  65 

o2  + 12  o2 
e 2 

*** 

3-  Perception 
Model 

2 

1014.08 

507.04 

<*♦ 

6i 

4-  Attack  Mode 

1 

10.  67 

10.67 

+ 12  °4 

12 

1 

3313.52 

3313.52 

0l  + 

6 o2 
12 

** 

13 

2 

183.29 

91.64 

02  + 
e 

4°13 

14 

1 

937.54 

937. 54 

o2  + 
e 

6014 

23 

2 

75.  60 

37.  80 

o2  + 
e 

4 °23 

24 

1 

60. 18 

60. 18 

o2  + 
e 

6 024 

34 

2 

123.  08 

61.  54 

o2  + 
e 

4 °34 

Residual 

9 

3505. 23 

389.47 

o2 

e 

Total 

23 

28719.79 

* Significant  at  a - 0. 10 
**  Significant  at  a - 0.05 
***  Significant  at  o.  = 0.01 


Figure  F.  1 . — Analysis  of  Variance  for  Subject  6 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 

Mean  Squares  Significance 

1 -Block 

1 

66.  67 

66.  67 

o2  + 12  o? 
e l 

2-  Route 

1 

2.  67 

2.  67 

o2  + 12  o2 
e 2 

3_  Perception 
Model 

2 

40.08 

20.04 

°i  + 

6 02 

4- Attack  Mode 

1 

4. 17 

4. 17 

o2  + 12  o2 
e 4 

12 

1 

4.17 

4. 17 

9 

0"  + 
e 

6°212 

13 

2 

308. 58 

154. 29 

°e  + 

40?3 

14 

1 

2.67 

2.  67 

°e  + 

6oL 

23 

2 

322.  58 

161.29 

o2  + 
e 

4 °23 

24 

1 

0.00 

0.00 

o2  + 
e 

6 °24 

34 

2 

273. 58 

136. 79 

o2  + 
e 

4 °34 

Residual 

9 

936. 66 

104. 07 

o2 

e 

Total 

23 

1961. 83 

* Significant  at  a - 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.  01 

Figure  F.  2 . — Analysis  of  Variance  for  Subject  7 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1 -Block 

1 

950. 04 

950.04 

a2  + 12  of 
C 1 

2- Route 

1 

2542. 04 

2542. 04 

4 4 12  °2 

** 

3-  Perception 
Model 

2 

3729. 25 

1864. 63 

6°3 

** 

4- Attack  Mode 

1 

126.  04 

126.  04 

o|  + 12  o2 

12 

1 

3675.  38 

3675. 38 

4* 

6 °12 

** 

13 

2 

370.09 

185. 04 

4* 

4 °13 

14 

1 

26.  04 

26.  04 

4* 

60u 

23 

2 

1336. 59 

668.  29 

4* 

4 °23 

24 

1 

92.  04 

92.  04 

a2  + 
e 

6024 

34 

2 

176.  58 

88.  29 

4 034 

Residual 

9 

3293.  55 

365.95 

o2 

e 

Total 

23 

16317. 62 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.  01 


Figure  F.  3 . — Analysis  of  Variance  for  Subject  8 


I 

I 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 

Mean  Squares  Significance 

1 -Block 

1 

24.  00 

24.  00 

°e  + 12  °f 

2-  Route 

1 

3700. 17 

3700.17 

o2  + 12  o2  * 

3-  Perception 
Model 

2 

1785. 33 

892. 67 

°i  + 

6 °3 

4-  Attack  Mode 

1 

1014. 00 

1014. 00 

4 + 12  4 

12 

1 

216.  00 

216.  00 

«!♦ 

60  12 

13 

2 

444. 00 

222.  00 

4* 

ian 

14 

1 

337.  50 

337.  50 

4 + 

6°14 

23 

2 

1017.  34 

508.  67 

4* 

4 023 

24 

1 

322. 67 

322.  67 

2 

°e  + 

6 °24 

34 

2 

756.  00 

378.00 

4* 

4 °34 

Residual 

9 

7522. 81 

835. 87 

4 

Total 

23 

17139.79 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a - 0.  01 


Figure  F.  4 . — Analysis  of  Variance  for  Subject  11 


400 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1 -Block 

1 

3.38 

3.  38 

2 9 

o%  + 12  cr 

2-  Route 

1 

2501.04 

2501.04 

0§  + 12  o22 

3-  Perception 
Model 

2 

7905. 33 

3952. 67 

ol+  6 a2 

*** 

4- Attack  Mode 

1 

693. 37 

693.  37 

9 2 

Op  + 12  o . 
e 4 

12 

1 

900.37 

900. 37 

o\  + 6 o22 

13 

2 

4107. 00 

2053. 50 

0§+  4 °13 

14 

1 

532. 04 

532.  04 

°i+  «°2U 

23 

2 

533. 34 

266. 67 

+ 4 023 

24 

1 

84.  38 

84.38 

4*  6 °24 

34 

2 

300. 00 

150. 00 

4 034 

Residual 

9 

4068. 67 

452.  07 

0 1 

Total 

23 

21628.  91 

* Significant  at  a = 0. 10 
**  Significant  at  a - 0. 05 
***  Significant  at  a = 0.  01 

Figure  F.  5 . — Anatysis  of  Variance  for  Subject  12 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1 -Block 

1 

170.67 

170. 67 

°e  + 12  °\ 

2-  Route 

1 

1.  50 

1.50 

o2  + 12  a2 

3-  Perception 
Model 

2 

9752.  58 

4876. 29 

ai  + 

6 °3 

** 

4- Attack  Mode 

1 

1.50 

1.  50 

4 * 12 

12 

1 

2480.  67 

2480.  67 

4* 

6 022 

* * 

13 

2 

12335.09 

6167.  54 

4* 

* * * 

14 

1 

10.  67 

10.  67 

4* 

60u 

23 

2 

391.75 

195.  87 

4* 

40L 

24 

1 

1.50 

1.  50 

6°24 

34 

2 

2784. 25 

1392. 12 

4* 

4 °34 

Residual 

9 

4319.16 

479.91 

4 

Total 

23 

36557.  79 

* Significant  at  a - 0. 10 
**  Significant  at  a = 0.05 
***  Significant  at  a = 0. 01 


Figure  F.  6 — Analysis  of  Variance  for  Subject  13 
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Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Square 

1 -Route 

1 

310.08 

310.08 

o|  + 6 o2 

2 -Perception 
Model 

2 

1029.17 

514.  58 

3- Attack  Mode 

1 

374.  08 

374. 08 

°e  + 6 °3 

12 

2 

155. 17 

77.58 

°e  + 2 

13 

1 

1220.08 

1220.08 

°e  + 3 

23 

2 

510.17 

255. 08 

+ 2 °23 

Residual 

2 

820.  16 

410.08 

4 

Total 

11 

4418.91 

Significance 


* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.  01 


Figure  F.  7 —Analysis  of  Variance  for  Subject  6 
(Trial  Two  Only) 
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Source  of  Degrees  of 

Variation  Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

1- Route 

1 

6.  75 

6.  75 

o|  + 6 oj 

2 -Perception 
Model 

2 

193. 50 

96.  75 

oi  + 4»2 

3- Attack  Mode 

1 

0.08 

0.08 

°l  * « °! 

12 

2 

303. 50 

196.  75 

°e  * 2 °i2 

13 

1 

30.08 

30.08 

2 2 

cC  + 3 o* 
e 13 

23 

2 

170. 17 

85.08 

°e  + 2 o223 

Residual 

2 

50. 17 

25.  08 

4 

Total 

11 

844. 25 

* Significant  at  a 
**  Significant  at  a 
***  Significant  at  a 

= 0.10 
= 0.05 
= 0.01 

Figure  F.  8 . — Analysis  of  Variance  for  Subject  7 
(Trial  Two  Only) 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

1- Route 

1 

52.08 

52.08 

o|  + 6 o2 

2 -Perception 
Model 

2 

879. 17 

439.  58 

°e  + 4 °2 

3- Attack  Mode 

1 

18.  75 

18.75 

°e  + 6 °3 

12 

2 

279. 17 

139.58 

°e  + 2 0i2 

13 

1 

752. 08 

752.  08 

4 * 3 <4 

23 

2 

262. 50 

131.25 

°e  + 2 4) 

Residual 

2 

379. 17 

189.58 

<4 

Total 

11 

2622. 92 

Significance 


* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0. 01 


Figure  F.  9 . — Analysis  of  Variance  for  Subject  8 
(Trial  Two  Only) 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 

Mean  Squares  Significance 

1 - Route 

1 

1064. 08 

1064.  08 

+ 6 °i 

2 -Perception 
Model 

2 

1880. 67 

940. 33 

4 + 4 °2 

3- Attack  Mode 

1 

1260.75 

1260. 75 

°e  ♦ « 0 3 

12 

2 

1248.  67 

624. 33 

°e  + 2 °l2 

13 

1 

1140.75 

1140.75 

4 * 3 o23 

23 

2 

134. 00 

67.  00 

4 * 2 <4 

Residual 

2 

482. 00 

241. 00 

4 

Total 

11 

7210.90 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0.  01 

Figure  F.  10. — Analysis  of  Variance  for  Subject  11 
(Trial  Two  Only) 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1- Route 

1 

3201.33 

3201. 33 

o|  + 6 o2 

** 

2 -Perception 
Model 

2 

308.17 

154. 08 

°e  + 4 °2 

3- Attack  Mode 

1 

5.  33 

5.  33 

4 + 6 °1 

12 

2 

88.17 

44.08 

4 * 2 °12 

13 

1 

705. 33 

705. 33 

4 * 2 °?3 

★ 

23 

2 

20.17 

10.08 

4 * 2 °23 

Residual 

2 

160.17 

80.08 

4 

Total 

11 

4488.  66 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0. 05 
***  Significant  at  a = 0.  01 


Figure  F.  11. — Analysis  of  Variance  for  Subject  12 
(Trial  Two  Only) 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1 -Route 

1 

1180.08 

1180.08 

o|  + e °2 

** 

2 -Perception 
Model 

2 

16750. 17 

8375.  08 

°!*402 

*** 

3- Attack  Mode 

1 

10.08 

10.08 

* « 4 

12 

2 

230. 17 

115.08 

* 2 °12 

13 

1 

10.08 

10.08 

°l  - 3 0 

23 

2 

50. 17 

25.08 

°e  + 2 °23 

Residual 

2 

50. 15 

25.  08 

4 

Total 

11 

18280.89 

* Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***■  Significant  at  a = 0.  01 


Figure  F.  12. — Analysis  of  Variance  for  Subject  13 
(Trial  Two  Only) 


ins 


Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 

Mean  Squares  Significance 

1-Block 

1 

5.  33 

5.  33 

of  + 24  of  + 12  0^ 

2- Route 

1 

1950.75 

1950. 75 

4 + 24  o22  ♦ 12  a223 

3- Subject 

1 

3675.  00 

3675.  00 

o 9 

at  + 24  o * *** 

e 3 

4- Perception 
Model 

2 

1107.  54 

553.  77 

4 * I®  o2  + 8 

5-AttackMode 

1 

574.  08 

574.  08 

of  + 24  of  + 12  of 
c 5 35 

12 

1 

80.08 

80.08 

4 * 12 

13 

1 

85.  33 

85.  33 

4 + 12 

14 

2 

744.  29 

372. 15 

4*  •<■*, 

15 

1 

140.08 

140.08 

of  + 12  o2 
e 15 

23 

1 

1752. 09 

1752. 09 

9 2 

°e  * I2  >>23 

24 

2 

529.  63 

264.  81 

4* 

25 

1 

161.  33 

161. 33 

°i  + 12  °25 

34 

2 

717.  88 

358. 94 

4*  » "34 

35 

1 

444.  08 

444. 08 

4 * 12  °35 

45 

2 

787. 79 

393. 90 

+ 8°45 

Residual 

27 

10021. 33 

371.16 

4 

Total 

47 

22776.  59 

* Significant  at  a = 0. 10 
**  Significant  at  a - 0.05 
***  Significant  at  cv  0.01 

Figure  F.I3. — Analysis  of  Variance  for  Subjects  7 and  11 
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Source  of 

Degrees  of 

Sums  of 

Mean 

Expected 

Variation 

Freedom 

Squares 

Squares 

Mean  Squares  Significance 

1- Block 

1 

420. 08 

420. 08 

4 + 24  o2  + 12  0^3 

2- Route 

1 

0.  08 

0.  08 

°1  + ^4  °2  + 12  °23 

3- Subject 

1 

3300. 08 

3300.08 

4 + 24  a2 

** 

4- Perception 
Model 

2 

388. 79 

194. 40 

4 + 16  °4  + « °34 

5-Attack  Mode 

1 

705. 33 

705. 33 

4 + 24  °5  + 12  °35 

12 

1 

4107. 00 

4107. 00 

4 + 12  °?2 

*** 

13 

1 

533. 33 

533. 33 

°e  4 12  °13 

14 

2 

1008. 29 

504. 15 

4 4 8 024 

15 

1 

396. 75 

396.  75 

4 4 12  0^ 

23 

1 

5043.  00 

5043.  00 

o|  + 12  a2 
e 23 

* 

*** 

24 

2 

1738.29 

869. 15 

4*  *°L 

25 

1 

0.  08 

0.08 

4 * 12  4 

34 

2 

11245. 78 

5622. 89 

4*  « 4 

*** 

35 

1 

114.  08 

114.08 

4 4 I2  °35 

45 

2 

115. 79 

57.90 

4 4 8 4 

Residual 

27 

12129.85 

449. 25 

4 

Total 

47 

41246.  58 

* Significant  at  a = 

0. 10 

**  Significant  at  a = 

0.  05 

***  Significant  at  a = 

0.01 

Figure  F.  14.  — Analysis  of  Variance  for  objects  8 and  12 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

1-  Route 

1 

620.  17 

620. 17 

°I  + 12  + 6 of 2 

2-  Subject 

1 

1320. 17 

1320. 17 

°e  + 12  °2 

s-  tepUo" 

2 

1634. 08 

817.04 

o2+  8o2  + 4 o23 

4-  Attack  Mode 

1 

640. 67 

640.  67 

°e  + I2  °4  + 6 °24 

12 

1 

450.  67 

450.  67 

°e+  ® °12 

13 

2 

143.  58 

71.79 

°l+  4 °13 

14 

1 

400. 17 

400. 17 

°I+  6o14 

23 

2 

440. 08 

220. 04 

°e+  4 °23 

24 

1 

620. 17 

620. 17^ 

°l+  6 °24 

34 

2 

289.  08 

144.  54 

2 , .2 
°e  + 4 °34 

Residual 

9 

2816. 50 

312.  94 

o2 

e 

Total 

23 

9375.  31 

* Significant  at  ci  = 0. 10 
**  Significant  at  a = 0.05 
***  Significant  at  a = 0.  01 


Figure  F.15. — Analysis  of  Variance  for  Subjects  7 and  11 
(Trial  Two  Only) 


Significance 
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Source  of 
Variation 

Degrees  of 
Freedom 

Sums  of 
Squares 

Mean 

Squares 

Expected 
Mean  Squares 

Significance 

1-  Route 

1 

2035. 04 

2035. 04 

of + 12  of+6of2 

2-  Subject 

1 

3243.  37 

3243.  37 

of  + 12  of 

*** 

^fia°e!pti0n 

2 

74.  08 

37.04 

°e  + 

803  + 4023 

4-  Attack  Mode 

1 

22.  04 

22.  04 

of  + 12  of  + 6 of. 

12 

1 

1218.  38 

1218.38 

°e  + 

8 °12 

* * * 

13 

2 

231. 58 

115. 79 

4°?3 

14 

1 

1457. 04 

1457. 04 

<*♦ 

+ ** 

23 

2 

1113.25 

556.  62 

°i  + 

4°l> 

** 

24 

1 

2.  04 

2.  04 

°1  + 

6 °24 

34 

2 

127.  58 

63.  79 

o ;♦ 

-L 

Residual 

9 

830. 55 

92.  28 

o2 

e 

Total 

23 

10354. 94 

♦ Significant  at  a = 0. 10 
**  Significant  at  a = 0.  05 
***  Significant  at  a = 0.  01 


Figure  F.  16. — Analysis  of  Variance  for  Subjects  8 and  12 
(Trial  Two  Only) 
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Figure  F.  17.  —Plot  of  Block  Effect  by  Subject  (Block  One 
Phase  II.  Block  Two  - Trial  Two) 
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Figure  F.  20.  — Plot  of  Attack  Mode  Effect  by  Block  and  Subject 
(Block  One  - Phase  II,  Block  Two  - Trial  Two) 
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Figure  F.  21.  — Plot  of  Perception  Model  Effect  by  Route  and 
Subject  (Trial  Two  Only) 
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Figure  F.  23.  —Plot  of  Perception  Model  Effect  by  Attack  Mode 
and  Subject  (Trial  Two  Only) 
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APPENDIX  G 

A PRELIMINARY  EXPERIMENT 
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As  preparation  for  the  Phase  I and  Phase  II  Experiments  reported  in 
Chapters  3 and  4,  a preliminary  experiment  was  conducted.  The  purpose  of  this 
experiment  was  to  investigate  the  procedures  that  were  being  considered  for 
use  during  Phase  I and  Phase  II.  It  was  desired  that  the  validity  of  these  pro- 
cedures be  established  before  they  be  depended  upon  to  produce  results  in  a 
production  environment. 

The  subject  group  used  during  the  preliminary  experiment  was  selected 
from  a group  of  volunteers  who  were  members  of  an  air  cavalry  troop  of  the 
Ohio  Air  National  Guard  stationed  at  Columbus,  Ohio.  All  subjects  had  had 
significant  combat  experience  as  gunship  pilots  in  Vietnam. 

The  portion  of  the  preliminary  experiment  corresponding  to  Phase  I 
was  conducted  at  the  Ohio  National  Guard  Armory  on  December  4,  1971.  Out 
of  eight  subjects  selected  to  participate , three  were  unable  to  attend  the  session 
at  the  appointed  hour.  Consequently,  they  were  dropped  from  further  considera- 
tion. Furthermore,  of  the  remaining  five  subjects,  two  were  required  to  leave 
before  they  had  completed  even  the  threat  evaluation  phase  of  the  experiment. 
Thus,  only  three  subjects  actually  completed  the  entire  Phase  I Experiment. 

During  Phase  I the  procedures  used  were  essentially  those  described  in 
Chapter  3 and  Appendix  B.  There  were  two  important  differences,  however. 
First,  <v>  particular  effort  was  made  to  describe  a tactical  scenario  to  which  the 
subjects  could  relate.  It  was  subsequently  discovered  that  the  subjects  found  it 
very  difficult  to  render  judgments  under  certain  circumstances  unless  they  could 
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render  these  judgments  in  the  context  of  a specific  scenario.  For  example,  the 
subjects  could  not  quote  an  unexposed  travel  time  that  they  would  find  equivalent 
to  a specified  exposed  threat  situation  unless  they  were  apprised  of  the  tactical 
scenario  in  which  the  decision  would  be  made.  A detailed  scenario  description 
was  incorporated  into  the  Phase  I Experiment  as  a result  of  this  finding. 

Second,  the  preliminary  experiment  included  a procedure  that  was  later 
dropped  from  the  Phase  I session  plan.  The  procedure  was  designed  to  deter- 
mine whether  or  not  the  assumption  of  valuewise  independence  with  respect  to 
exposure  to  multiple  enemy  weapons  was  justified.  It  was  found  that  without 
exception  the  subjects  exhibited  such  behavior.  Since  it  was  desired  to  shorten 
the  Phase  I Experiment  anyway,  it  was  decided  that  the  procedure  could  be 
dropped  with  little  loss  in  information. 

Procedures  employed  during  the  portion  of  the  preliminary  experiment 
corresponding  to  the  Phase  II  Experiment  were  also  similar  to  those  described 
in  Chapter  4 and  Appendix  D.  However,  during  the  preliminary  experiment  only 
three  alternative  routes  were  prepared  for  each  of  the  route  plans  constructed  by 
the  subjects.  The  three  alternatives  differed  from  one  another  only  in  the  threat 
perception  model  that  was  used  to  measure  threat.  In  each  situation  the  attack 
mode  employed  was  that  selected  by  the  subject  himself.  Consequently,  no  data 
were  produced  that  could  be  used  in  measuring  an  attack  mode  effect. 

Since  only  three  alternatives  were  to  be  considered  by  a subject  for  each 
route  selection  situation,  the  instructions  for  ranking  and  rating  the  alternatives 
were  different  from  those  employed  during  the  Phase  II  Experiment.  Rather  than 
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rank  and  rate  each  alternative  relative  to  his  own  route  plan,  each  subject  was 
instructed  to  consider  all  four  routes  simultaneously  and  to  rate  the  best  plan 
at  100.  Then  the  subject  was  instructed  to  rate  the  next  best  plan  on  a scale  of 
0 - 100  using  the  rules  discussed  in  Chapter  4.  The  subject  was  instructed  to 
continue  until  all  route  alternatives  had  been  rated.  As  discussed  in  Chapter  4 
this  procedure  could  not  be  repeated  during  Phase  II  because  of  the  greater 
number  of  alternatives  to  be  considered. 

A final  difference  between  the  preliminary  experiment  and  the  Phase  II 
Experiment  arose  because  the  National  Guard  unit  to  which  the  subjects  belonged 
met  only  once  a month.  Thus,  it  was  necessary  to  conduct  the  Phase  II  Experi- 
ment by  mail.  This  turned  out  to  be  a very  unfortunate  circumstance. 

On  January  17,  1972,  each  subject  was  mailed  an  experimental  supply 

■ 

package  similar  in  content  to  that  which  was  prepared  for  the  subjects  partici- 

I 

pating  in  the  Trial  Two  Experiment  (See  Appendix  F).  However,  of  the  three 
subjects  still  participating,  only  one  was  prompt  in  returning  his  completed 
experimental  package.  After  repeated  phone  calls  a second  subject  returned 
his  package  in  about  a month.  However,  the  third  subject  never  did  reply, 
although  he  promised  to  on  two  different  occasions.  This  experience  proved 
very  valuable  later  when  it  was  necessary  to  conduct  the  Trial  Two  Experiment 
by  mail  as  discussed  in  Chapter  5.  Elaborate  precautions  were  taken  to  insure 
that  the  subjects  responded  as  requested. 
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